Re: Allow file inclusion in pg_hba and pg_ident files

On Tue, Nov 22, 2022 at 05:20:01PM +0900, Michael Paquier wrote:
> + /* XXX: this should stick to elevel for some cases? */
> + ereport(LOG,
> + (errmsg("skipping missing authentication file \"%s\"",
> + inc_fullname)));
> Should we always issue a LOG here? In some cases we use an elevel of
> DEBUG3.

And here I think that we need to use elevel. In hba.c, this would
generate a LOG while hbafuncs.c uses DEBUG3, leading to less noise in
the log files.

> So, what do you think about something like the attached? I have begun
> a lookup at the tests, but I don't have enough material for an actual
> review of this part yet. Note that I have removed temporarily
> process_included_authfile(), as I was looking at all the code branches
> in details. The final result ought to include AbsoluteConfigLocation(),
> open_auth_file() and tokenize_auth_file() with an extra missing_ok
> argument, I guess ("strict" as proposed originally is not the usual
> PG-way for ENOENT-ish problems). process_line should be used only
> when we have no err_msg, meaning that these have been consumed in some
> TokenizedAuthLines already.

This, however, was still brittle in terms of memory handling.
Reloading the server a few hundred times proved that this was leaking
memory in the tokenization. This becomes quite simple once you switch
to an approach where tokenize_auth_file() uses its own memory context,
while we store all the TokenizedAuthLines in the static context. It
also occurred to me that we can create and drop the static context
holding the tokens when opening the top-root auth file, aka with a
depth at 0. It may be a bit simpler to switch to a single-context
approach for all the allocations of tokenize_auth_file(), though I
find the use of a static context much cleaner for the inclusions with
@ files when we expand an existing list.

The patch can be split, again, into more pieces. Attached 0001
reworks the memory allocation contexts for the tokenization and 0002
is the main feature. As of things are, I am quite happy with the
shapes of 0001 and 0002. I have tested quite a bit its robustness,
with valgrind for example, to make sure that there are no leaks in the
postmaster (with[out] -DEXEC_BACKEND). The inclusion logic is
refactored to be in a state close to your previous patches, see
tokenize_inclusion_file().

Note that the tests are failing for some of the Windows CIs, actually,
due to a difference in some of the paths generated vs the file paths
(aka c:\cirrus vs c:/cirrus, as far as I saw).
--
Michael