Re: Roadmap for a Win32 port

I think SGI gets amazing performance because they have very good (efficient)
synchronisation primitives on SGI. Some proprietary light-weight mutexes.
Using threaded or mixed model just by itself is not going to do a miracle.
Threads will save you some context switch time, but that will probably
translate into lower CPU usage rather than performance boost. And if your
mutexes are not fast or awkwardly implemented (say Linux), it might be even
worse. Apache is not all that fast on Linux as on SGI, whatever model you
chose. I also doubt that purely threaded model would be slower than mixed
one.

Now about the AIO model. It is useful when you need to do something else
while I/O requests are being processed as long as platform does it in some
useful way. If all you can do is to submit requests and keep sitting in
select/poll then AIO does not buy you anything you can't get by just using
threaded model. However, if you can tag the requests and set up
notifications, then few I/O threads could handle relatively large number of
requests from different clients. Note, this means you don't have any
association between clients and servers at all, there is pool of generic I/O
threads which serve requests from whoever they come. It saves system
resources and scales very well. It also provides interesting possibilities
for fault recovery - since handlers are generic all the state information
would have to be kept in some kind of global context area. That area can be
saved into persistent memory or dumped onto disk and *recovered* after a
forced restart. Server and library could be designed in such a way that
clients may continue where they left with a recoverable error.

In POSIX AIO model you can tag requests and set up notifications via
synchronous signals. You wait for them *synchronously* in 'waiter' thread
via sigwaitinfo() and avoid the headache of asynchronous signals hitting you
any time... Unfortunately on some platforms (Solaris) the depth of
synchronous signal queue is fixed at magic value 32 (and not adjustable).
This may not be a problem if you're sure that waiting thread will be able to
drain the queue faster than it gets filled with notifications... but I'm not
sure there is a portable way to guarantee that, so you need to check for
overloads and handle them... that complicates things. On Solaris you also
need a mile of compiler/linker switches to even get this scheme to work and
I am afraid other platforms may not support it at all (but then again, they
may not support AIO to begin with).

And speaking about getting best of all worlds. Note how Apache spent nearly
3 years developing their portable Multi-Processing Modules scheme. What they
got for that is handful of models neither of which perform noticeably better
than original pre-fork() model. Trying to swallow all possible ways to
handle things on all possible platforms usually does not produce very fast
code. It tends to produce very complex code with mediocre performance and
introduces extra complexity into configuration process. If you consider all
that was done mostly to support Win32, one might doubt if it was worth the
while.

What I am trying to say is, extra complexity in model to squeeze few percent
of performance is not a wise investment of time and efforts. On Win32 you
don't really compete in terms of performance. You compete in terms of
easyness and features. Spend 3 years trying to support Windows and Unix in
most optimal way including all subvariants of Unix ... meanwhile MSFT will
come up with some bundled SQL server. It probably will have more features
since they will spend time doing features rather than inventing a model to
support gazillion of platforms. Chances are, it will be faster too - due to
better integration with OS and better compiler.

I am not in position to tell you what to do guys. But if I was asked, I'd
say supporting Win32 is only worth it if it comes as a natural result of a
simple, coherent and uniform model applied to Unix. Threaded model may not
have as much inherent stability as forked/mixed, but it has inherent
simplicity and better Unix/Windows/BeOS portability. It can be done faster
and simpler code will make work on features easier.

Regards,
- Igor

"There are 2 ways to design an efficient system - first is to design it so
complex that there are no obvious deficiencies, second is to design it so
simple that there are obviously no deficiencies. Second way is much harder"
(author unknown to me)

----- Original Message -----
From: "Neil Conway" <nconway(at)klamath(dot)dyndns(dot)org>
To: "Jon Franz" <coventry(at)one(dot)net>
Cc: <pgsql-hackers(at)postgresql(dot)org>
Sent: Wednesday, June 05, 2002 7:05 PM
Subject: Re: [HACKERS] Roadmap for a Win32 port

> On Wed, 5 Jun 2002 18:50:46 -0400
> "Jon Franz" <coventry(at)one(dot)net> wrote:
> > One note: SGI developers discovered they could get amazing performance
using
> > as hybrid threaded and forked-process model with apache - we might want
to
> > look into this. They even have a library for network-communication
> > utilizing thier 'state threads' model.
>
> I think ST is designed for network I/O-bound apps -- last I checked,
> disk I/O will still block an entire ST process. While you can get around
> that by using another process to do disk I/O, it sounds like ST won't be
> that useful.
>
> However, Chris KL. (I believe) raised the idea of using POSIX AIO for
> PostgreSQL. Without having looked into it extensively, this technique
> sounds promising. Perhaps someone who has looked into this further
> (e.g. someone from Redhat) can comment?
>
> Cheers,
>
> Neil
>
> --
> Neil Conway <neilconway(at)rogers(dot)com>
> PGP Key ID: DB3C29FC
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Don't 'kill -9' the postmaster
>