Re: Experimental patch for inter-page delay in VACUUM

Greg Stark wrote:

> Shridhar Daithankar <shridhar_daithankar(at)myrealbox(dot)com> writes:
>
>
>>If the parent postmaster mmaps anonymous memory segments and shares them with
>>children, postgresql wouldn't be dependent upon any kernel resourse aka
>>shared memory anymore.
>
>
> Anonymous memory mappings aren't shared, at least not unless you're talking
> about creating posix threads. That's just not how you create shared mappings
> using mmap.
>
> There is a way to create shared mappings using mmap, but it's exactly what you
> say you don't want to do -- you use file mappings.
>
> Using mmap postgres could allocate as much shared memory as it needs whenever
> it needs it. You create a file the size of the mapping you want, you mmap it
> with MAP_SHARED, then you arrange to have any other backends that want access
> to it to mmap it as well.

Yes. It occurred to me in the morning. For sure, a good night sleep helps..
>
> I'm not sure why you say you don't want to map files. If you're afraid it will
> cause lots of i/o as the system tries to flush these writes, well, in theory
> that's up to the kernel to avoid. On systems where the kernel does poorly at
> this there are tools like MAP_LOCK/mlock/shmfs that might trick it into doing
> a better job.

I didn't have any file in my first post because I saw it as unnecessary. However
my guess is IO caused by such file would not be much. How muh shared bufffers
postgresql would be using anyways? 100MB? 200MB?

On the bright side, system will automatically sync the shared buffers
periodically. It is like taking snapshot of shaerd buffers. Could be good at
debugging.

If the IO caused by such a shared memory image is really an issue for somebody,
they can just map the file on a ramdrive.

Actaully I would say that would be a good default approach. Use mmaped file over
RAM drive as shared buffers. Just wondering if it can be done programmatically.

> Actually I've been wondering how hard it would be to avoid this whole
> double-buffering issue and having postgres mmap the buffers it wants from the
> data files. That would avoid the double-buffering entirely including the extra
> copy and memory use. But it would be a major change to a lot of core stuff.
> And it be tricky to ensure WAL buffers are written before data blocks.

Yes. I understand mmap is not adequete for WAL and other transaction syncing
requirement.

Bye
Shridhar