Re: Shared memory size computation oversight?

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Georgios <gkokolatos(at)protonmail(dot)com>
Cc: Michael Paquier <michael(at)paquier(dot)xyz>, Julien Rouhaud <rjuju123(at)gmail(dot)com>, Julien Rouhaud <julien(dot)rouhaud(at)free(dot)fr>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, "robertmhaas(at)gmail(dot)com" <robertmhaas(at)gmail(dot)com>
Subject: Re: Shared memory size computation oversight?
Date: 2021-03-04 17:53:54
Message-ID: 1374257.1614880434@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

I spent a bit of time looking into this. Using current HEAD, I
instrumented CreateSharedMemoryAndSemaphores to log the size estimates
returned by the various estimation subroutines, plus the shared memory
space actually consumed (i.e, the change in ShmemSegHdr->freeoffset)
by the various shared memory initialization functions. There were only
two estimates that were way off: LockShmemSize requested 1651771 more
bytes than InitLocks actually consumed, and PredicateLockShmemSize
requested 167058 more bytes than InitPredicateLocks consumed. I believe
both of those are intentional, reflecting space that may be eaten by the
lock tables later.

Meanwhile, looking at ShmemSegHdr->freeoffset vs ShmemSegHdr->totalsize,
the actual remaining shmem space after postmaster startup is 1919488
bytes. (Running the core regression tests doesn't consume any of that
remaining space, btw.) Subtracting the extra lock-table space, we have
100659 bytes to spare, which is as near as makes no difference to the
intended slop of 100000 bytes.

My conclusion is that we don't need to do anything, indeed the proposed
changes will probably just lead to overestimation.

It's certainly possible that there's something amiss somewhere. These
numbers were all taken with out-of-the-box configuration, so it could be
that changing some postgresql.conf entries would expose that some module
is not scaling its request correctly. Also, I don't have any extensions
loaded, so this proves nothing about the correctness of any of those.
But it appears to me that the general scheme is working perfectly fine,
so we do not need to complicate it.

regards, tom lane

Attachment Content-Type Size
shmem-consumption.txt text/plain 1.9 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Andres Freund 2021-03-04 17:56:50 Re: buildfarm windows checks / tap tests on windows
Previous Message Julien Rouhaud 2021-03-04 17:52:28 Re: Shared memory size computation oversight?