Re: Estimating HugePages Requirements?

On Thu, Sep 9, 2021 at 11:53 PM Bossart, Nathan <bossartn(at)amazon(dot)com> wrote:
>
> On 9/8/21, 9:19 PM, "Michael Paquier" <michael(at)paquier(dot)xyz> wrote:
> > FWIW, I don't have an environment at hand these days to test properly
> > 0001, so this will have to wait a bit. I really like the approach
> > taken by 0002, and it is independent of the other patch while
> > extending support for postgres -c to provide the correct runtime
> > values. So let's wrap this part first. No need to send a reorganized
> > patch set.
>
> Sounds good.
>
> For 0001, the biggest thing on my mind at the moment is the name of
> the GUC. "huge_pages_required" feels kind of ambiguous. From the
> name alone, it could mean either "the number of huge pages required"
> or "huge pages are required for the server to run." Also, the number
> of huge pages required is not actually required if you don't want to
> run the server with huge pages. I think it might be clearer to
> somehow indicate that the value is essentially the size of the main
> shared memory area in terms of the huge page size, but I'm not sure
> how to do that concisely. Perhaps it is enough to just make sure the
> description of "huge_pages_required" is detailed enough.
>
> For 0002, I have two small concerns. My first concern is that it
> might be confusing to customers when the runtime GUCs cannot be
> returned for a running server. We have the note in the docs, but if
> you're encountering it on the command line, it's not totally clear
> what the problem is.
>
> $ postgres -D . -C log_min_messages
> warning
> $ postgres -D . -C shared_memory_size
> 2021-09-09 18:51:21.617 UTC [7924] FATAL: lock file "postmaster.pid" already exists
> 2021-09-09 18:51:21.617 UTC [7924] HINT: Is another postmaster (PID 7912) running in data directory "/local/home/bossartn/pgdata"?
>
> My other concern is that by default, viewing the runtime-computed GUCs
> will also emit a LOG.
>
> $ postgres -D . -C shared_memory_size
> 142
> 2021-09-09 18:53:25.194 UTC [8006] LOG: database system is shut down
>
> Running these commands with log_min_messages=debug5 emits way more
> information for the runtime-computed GUCs than for others, but IMO
> that is alright. However, perhaps we should adjust the logging in
> 0002 to improve the default user experience. I attached an attempt at
> that.
>
> With the attached patch, trying to view a runtime-computed GUC on a
> running server will look like this:
>
> $ postgres -D . -C shared_memory_size
> 2021-09-09 21:24:21.552 UTC [6224] FATAL: lock file "postmaster.pid" already exists
> 2021-09-09 21:24:21.552 UTC [6224] DETAIL: Runtime-computed GUC "shared_memory_size" cannot be viewed on a running server.
> 2021-09-09 21:24:21.552 UTC [6224] HINT: Is another postmaster (PID 3628) running in data directory "/local/home/bossartn/pgdata"?
>
> And viewing a runtime-computed GUC on a server that is shut down will
> look like this:
>
> $ postgres -D . -C shared_memory_size
> 142

Nothing fixing this ended up actually getting committed, right? That
is, we still get the extra log output?

And in fact, the command documented on
https://www.postgresql.org/docs/devel/kernel-resources.html doesn't
actually produce the output that the docs show, it also shows the log
line, in the default config? If we can't fix the extra logging we
should at least have our docs represent reality -- maybe by adding a
"2>/dev/null" entry? But it'd be better to have it not output that log
in the first place...

(Of course what I'd really want is to be able to run it on a cluster
that's running, but that was discussed downthread so I'm not bringing
that one up for changes now)

--
Magnus Hagander
Me: https://www.hagander.net/
Work: https://www.redpill-linpro.com/