Re: buildfarm server suddenly not talking to old SSL stacks?

On 07/17/2018 11:29 PM, Tom Lane wrote:
> Stefan Kaltenbrunner <stefan(at)kaltenbrunner(dot)cc> writes:
>> On 07/17/2018 10:14 PM, Tom Lane wrote:
>>> So for some reason, perl's https support is trying to bind to the IPv6
>>> address of buildfarm.postgresql.org, even though no IPv6 support is
>>> configured at all on this machine. I wonder how long that's been going
>>> on? Has anything about the machine's DNS entries changed recently?
>>> (Also, "ssh buildfarm.postgresql.org" binds to IPv4 just fine.)
>
>> I dont think there have been any recent changes on (DNS) v6 for
>> brentalia - afaiks in our internal revision control we have had v6 on
>> that box for at least 2 years now.
>> However could it be that whatever DNS resolver those boxes are using
>> just started to return AAAAs as well (the strsize in the strace output
>> is not large enough to see the actual response from the local resolver)
>
> The nameserver is one I run locally, and the only change it's seen lately
> is RHEL6's occasional security updates. I don't think that's where the
> issue came in.
>
> The full nameserver interaction is
>
> sendto(3, "\x21\x86\x01\x00\x00\x01\x00\x00\x00\x00\x00\x00\x09\x62\x75\x69\x6c\x64\x66\x61\x72\x6d\x0a\x70\x6f\x73\x74\x67\x72\x65\x73\x71\x6c\x03\x6f\x72\x67\x00\x00\x1c\x00\x01", 42, MSG_NOSIGNAL, NULL, 0) = 42
>
> recvfrom(3, "\x21\x86\x81\x80\x00\x01\x00\x01\x00\x00\x00\x00\x09\x62\x75\x69\x6c\x64\x66\x61\x72\x6d\x0a\x70\x6f\x73\x74\x67\x72\x65\x73\x71\x6c\x03\x6f\x72\x67\x00\x00\x1c\x00\x01\xc0\x0c\x00\x1c\x00\x01\x00\x00\x06\xc1\x00\x10\x20\x01\x48\x00\x15\x01\x00\x01\x00\x00\x00\x00\x00\x00\x02\x17", 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("127.0.0.1")}, [16]) = 70
>
> I don't have anything handy like wireshark installed on this machine, but
> I see the hex for buildfarm's IPv6 address in that response, and *not*
> the hex for its IPv4 address. Conversely, when I try the http: URL,
> I see a different query and only the IPv4 address in the response:
>
> sendto(3, "\xa8\x93\x01\x00\x00\x01\x00\x00\x00\x00\x00\x00\x09\x62\x75\x69\x6c\x64\x66\x61\x72\x6d\x0a\x70\x6f\x73\x74\x67\x72\x65\x73\x71\x6c\x03\x6f\x72\x67\x00\x00\x01\x00\x01", 42, MSG_NOSIGNAL, NULL, 0) = 42
>
> recvfrom(3, "\xa8\x93\x81\x80\x00\x01\x00\x01\x00\x00\x00\x00\x09\x62\x75\x69\x6c\x64\x66\x61\x72\x6d\x0a\x70\x6f\x73\x74\x67\x72\x65\x73\x71\x6c\x03\x6f\x72\x67\x00\x00\x01\x00\x01\xc0\x0c\x00\x01\x00\x01\x00\x00\x01\xd5\x00\x04\xae\x8f\x23\xd9", 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("127.0.0.1")}, [16]) = 58
>
> It looks like Perl is specifically asking for AAAA in preference to A
> records, but only for https:. Weird.

not really weird I think - the buildfarm uses LWP and for SSL support it
might use(iirc) either Crypt::SSLeay (older versions before unbundling
of lwp::protocol:https) or IO::Socket:SSL which has this in its docs:

"Please be aware that with the IPv6 capable super classes, it will look
first for the IPv6 address of a given hostname. If the resolver provides
an IPv6 address, but the host cannot be reached by IPv6, there will be
no automatic fallback to IPv4. To avoid these problems you can enforce
IPv4 for a specific socket by using the Domain or Family option with the
value AF_INET as described in IO::Socket::IP. Alternatively you can
enforce IPv4 globally by loading IO::Socket::SSL with the option
'inet4', in which case it will use the IPv4 only class IO::Socket::INET
as the super class."

So maybe removing the IO::Socket::INET6 superclass/package from the
system will get it working (or hacking the buildfarm script).

Stefan