Re: pgbench bug / limitation

Hi,

On 2020-06-05 13:32:35 -0400, Tom Lane wrote:
> Andres Freund <andres(at)anarazel(dot)de> writes:
> > On June 5, 2020 9:45:47 AM PDT, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> >> The idea that I vaguely had was to build our own array of socket FDs
> >> (bypassing the unnecessary de-duplication logic in FD_SET) and then
> >> call WaitForMultipleObjects() or similar directly.
>
> > IIRC WaitForMultiple* only supports 64 objects or such. Which might be problematic here.
>
> Ugh, so it does. I'd also just noted that its timeout resolution is
> only in msec, which is exactly why we want to use ppoll() not poll()
> here on Unix-oid OS's. So WaitForMultipleObjects() is out.
>
> I still suppose that select(2) is not a native API for Windows. Since
> we know that it can be made to support more than 64 FDs, it must not
> be built on top of WaitForMultipleObjects ... but then what *is* it
> built on?

I don't know. I only remembered this limitation because at some point I
was looking at making the latch code cope with large numbers of FDs
(e.g. for an append over FDWs). IIRC one basically needs helper threads
or something.

I'm fairly sure one can easily scale to large numbers of sockets on
windows by using completion based APIs, but that doesn't seem like a
realistic thing to use for pgbench anytime soon.

Greetings,

Andres Freund