Re: Providing anonymous mmap as an option of sharing memory

Tom Lane wrote:

> Shridhar Daithankar <shridhar_daithankar(at)myrealbox(dot)com> writes:
>
>>I covered only first point in my post. IMO it is not such a unsolvable
>>problem. If a postmaster crashes hard but leaves a backend running,
>>would it clean pid file etc? I don't think so. So if a postmaster can
>>start on a 'pid-clean' state, then it is guaranteed to be no childs
>>left around.
>
>
> And that helps how? The problem is to detect whether there are any
> children left from the old postmaster, when what you have to work from
> is the pid-file it left behind.

fine. We need shared memory for that. How about using 1 8K page just for
detecting that? We don't need to base shared memory model on that, right?

May be we can put clog in shared memory segment which would serve as process
counter and move shared buffers to mmap?

> In any case, you're still handwaving away the very real portability
> issues around mmap. Linux is not the universe, and Linux+BSD isn't
> either.

From the machines I can access here, following have anon and shared mmap..

[ost] ~> uname -a
SunOS host 5.8 Generic_108528-21 sun4u sparc SUNW,Sun-Fire-880 Solaris

[host] ~> uname -a
AIX host 1 5 0001A5CA4C00

[/home/user]uname -a
HP-UX host B.11.00 A 9000/785 2005950738 two-user license

Is it enough of support?

>
> We might still have considered it, despite the negatives, if anyone had
> been able to point to any actual *advantages* of mmap. There are none.
> Yes, the SysV shmem API is old and ugly and crufty, but it does what we
> need it to do.

1) Per database buffers

Postgresql does not perform well with large number of buffers. Say an
installation is configured for 100K buffers and have 5 databases. Now what would
happen if each of these databases get their own 100K buffers?

mmap can not expand shared memory without a server restart. The current
implementation of shared memory behaves the same way.

Rather than moving it to use shared memory as and when required, we could push
per database buffers to improve scalability.

I think of this.

1. Introduce parameter columns in pg_database, for shared memory size (to start
with) and number of live connections to that database. May be a callback to
daemon postmaster to reread configuration if possible. (In shared memory, may be?)

2. Provide start and stop server commands which essentially either let a
connection happen or not.

Now somebody modifies the buffer parameters for a database(Say via alter
database), all it has to do is disconnect and reconnect. If this is a first
connection to the database, the parent postmaster should reread the per database
parameters and force them. Same can happen with start/stop commands.

2) No more kernel mucking required.

Recent linux installations are provide sane enough default of SHMMAX but I am
sure plenty of folks would be glad to see that dependency go.

I also want to talk about mmap for file IO but not in this thread.

Shridhar