Re: Estimating HugePages Requirements?

On Fri, Sep 03, 2021 at 05:36:43PM +0000, Bossart, Nathan wrote:
> On 9/2/21, 6:46 PM, "Michael Paquier" <michael(at)paquier(dot)xyz> wrote:
> You bring up an interesting point about _PG_init(). Presently, you
> can safely assume that the data directory is locked during _PG_init(),
> so there's no need to worry about breaking something on a running
> server. I don't know how important this is. Most _PG_init()
> functions that I've seen will define some GUCs, request some shared
> memory, register some background workers, and/or install some hooks.
> Those all seem like safe things to do, but I wouldn't be at all
> surprised to hear examples to the contrary. In any case, it looks
> like the current ordering of these two steps has been there for 15+
> years.

Yeah. What you are describing here matches what I have seen in the
past and what we do in core for _PG_init(). Now extensions developers
could do more fancy things, like dropping things on-disk to check the
load state, for whatever reasons. And things could break in such
cases. Perhaps people should not do that, but it is no fun either to
break code that has been working for years even if that's just a major
upgrade.

+ * We skip this step if we are just going to print a GUC's value and exit
+ * a few steps down.
*/
- CreateDataDirLockFile(true);
+ if (output_config_variable == NULL)
+ CreateDataDirLockFile(true);

Anyway, 0002 gives me shivers.

> If this is a concern, one option would be to disallow running "-C
> shared_memory_size" on running servers. That would have to extend to
> GUCs like data_checksums and huge_pages_required, too.

Just noting this bit from 0003 that would break without 0002:
-$ <userinput>pmap 4170 | awk '/rw-s/ &amp;&amp; /zero/ {print $2}'</userinput>
-6490428K
+$ <userinput>postgres -D $PGDATA -C shared_memory_size</userinput>
+6339

>> 0001, that refactors the calculation of the shmem size into a
>> different routine, is fine as-is, so I'd like to move on and apply
>> it.
>
> Sounds good to me.

Applied this one.

Without concluding on 0002 yet, another thing that we could do is to
just add the GUCs. These sound rather useful on their own (mixed
feelings about huge_pages_required but I can see why it is useful to
avoid the setup steps and the backend already grabs this information),
particularly when it comes to cloned setups that share a lot of
properties.
--
Michael