Re: Roadmap for a Win32 port

Jan Wieck wrote:
> > One solution is to return to that for Win32 only, so instead of doing:
> >
> > initialization()
> > want for connection()
> > fork backend()
> >
> > we do for Win32:
> >
> > want for connection()
> > exec backend()
> > initialization()
>
> Summarizes pretty much what we discussed Monday on the phone.
> Except that the postmaster still has to initialize the shared
> memory and other stuff. It's just that the backends and
> helper processes need to reinitialize themself (attach).

Yes, obviously I simplified, and I do believe our optimizations are
helping on Unix. It is just that I think for Win32 the fork is more
harmful than removing those optimizations.

One thing that may not have been clear is that we don't need to play
with globals at all. We just pass whatever info we want to the child
via command-line arguments, rather than shared memory.

> > It wouldn't be hard to do. We would still do CreateProcess rather than
> > CreateThread, but it eliminates the fork/threading issues. We don't
> > know the database before the connection arrives, so we don't do a whole
> > lot of initialization.
>
> All I see so far is the reading of the postgresql.conf, the
> pg_hba.conf and the password files. Nothing fancy and the
> postmaster could easily write out a binary content only file
> that the backends then read, eliminating the parsing
> overhead.

Yes, that is clearly possible. Another option is to just write out a
no-comments, no-whitespace version of each file and just have the
backends read those. The advantage is that we can use the same code to
read them, and I don't think it would be any slower than a binary file.

> The bad news is that Tom is right. We did a terrible job in
> using the new side effect, that the shared memory segment is
> at the same address in all forked processes, after removing
> the need to reattach.
>
> In detail the XLog code, the FreeSpaceMap code and the
> "shared memory" hashtable code now use pointers, located in
> shared memory. For the XLog and FreeSpace code this is
> understandable, because they where developed under the fork()
> only model. But the dynahash code used offsets only until
> v7.1!
>
> All three (no claim that that's all) make it impossible to
> ever have someone attaching to the shared memory from the
> outside. So with these moves we made the shared memory a
> "Postmaster and children" only thing. Raises the question,
> why we need an IPC key at all any more.

Well, we could force shmat() to bind to the same address, but I suspect
that might fail in some cases.

> Anyhow, looks as if I can get that fork() vs. fork()+exec()
> feature done pretty soon. It'll be controlled by another
> Postmaster commandline switch. After cleaning up the mess I
> did to get it working quick, I'll provide a patch for
> discussion.

Yes, very little impact. We then need someone to do some Win32 timings
to see if things have improved. As Tom mentioned, we need some hard
numbers for these things. In fact, I would like a Win32 test that takes
our code and compares fork(), then exit(), with CreateProcess(), exit().
It doesn't have create a db session, but I would like to see some
timings to know what we are gaining. Heck, time CreateThread too and
let's see what that shows.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026