Chapter 27. Using pg_options

Note: Contributed by Massimo Dal Zotto

The optional file data/pg_options contains runtime options used by the backend to control trace messages and other backend tunable parameters. What makes this file interesting is the fact that it is re-read by a backend when it receives a SIGHUP signal, making thus possible to change run-time options on the fly without needing to restart Postgres. The options specified in this file may be debugging flags used by the trace package (backend/utils/misc/trace.c) or numeric parameters which can be used by the backend to control its behaviour. New options and parameters must be defined in backend/utils/misc/trace.c and backend/include/utils/trace.h.

For example suppose we want to add conditional trace messages and a tunable numeric parameter to the code in file foo.c. All we need to do is to add the constant TRACE_FOO and OPT_FOO_PARAM into backend/include/utils/trace.h:

/* file trace.h */
enum pg_option_enum {
    ...
    TRACE_FOO,                  /* trace foo functions */
    OPT_FOO_PARAM,              /* foo tunable parameter */

    NUM_PG_OPTIONS              /* must be the last item of enum */
};

and a corresponding line in backend/utils/misc/trace.c:

/* file trace.c */
static char *opt_names[] = {
    ...
    "foo",                      /* trace foo functions */
    "fooparam"                  /* foo tunable parameter */
};

Options in the two files must be specified in exactly the same order. In the foo source files we can now reference the new flags with:

/* file foo.c */
#include "trace.h"
#define foo_param pg_options[OPT_FOO_PARAM]

int
foo_function(int x, int y)
{
    TPRINTF(TRACE_FOO, "entering foo_function, foo_param=%d", foo_param);
    if (foo_param > 10) {
        do_more_foo(x, y);
    }
}

Existing files using private trace flags can be changed by simply adding the following code:

#include "trace.h"
/* int my_own_flag = 0; -- removed */
#define my_own_flag pg_options[OPT_MY_OWN_FLAG]

All pg_options are initialized to zero at backend startup. If we need a different default value we must add some initialization code at the beginning of PostgresMain. Now we can set the foo_param and enable foo trace by writing values into the data/pg_options file:

# file pg_options
...
foo=1
fooparam=17

The new options will be read by all new backends when they are started. To make effective the changes for all running backends we need to send a SIGHUP to the postmaster. The signal will be automatically sent to all the backends. We can also activate the changes only for a specific backend by sending the SIGHUP directly to it.

pg_options can also be specified with the -T switch of Postgres:

postgres options -T "verbose=2,query,hostlookup-"

The functions used for printing errors and debug messages can now make use of the syslog(2) facility. Message printed to stdout or stderr are prefixed by a timestamp containing also the backend pid:

#timestamp          #pid    #message
980127.17:52:14.173 [29271] StartTransactionCommand
980127.17:52:14.174 [29271] ProcessUtility: drop table t;
980127.17:52:14.186 [29271] SIIncNumEntries: table is 70% full
980127.17:52:14.186 [29286] Async_NotifyHandler
980127.17:52:14.186 [29286] Waking up sleeping backend process
980127.19:52:14.292 [29286] Async_NotifyFrontEnd
980127.19:52:14.413 [29286] Async_NotifyFrontEnd done
980127.19:52:14.466 [29286] Async_NotifyHandler done

This format improves readability of the logs and allows people to understand exactly which backend is doing what and at which time. It also makes easier to write simple awk or perl scripts which monitor the log to detect database errors or problem, or to compute transaction time statistics.

Messages printed to syslog use the log facility LOG_LOCAL0. The use of syslog can be controlled with the syslog pg_option. Unfortunately many functions call directly printf() to print their messages to stdout or stderr and this output can't be redirected to syslog or have timestamps in it. It would be advisable that all calls to printf would be replaced with the PRINTF macro and output to stderr be changed to use EPRINTF instead so that we can control all output in a uniform way.

The new pg_options mechanism is more convenient than defining new backend option switches because:

• we don't have to define a different switch for each thing we want to control. All options are defined as keywords in an external file stored in the data directory.

• we don't have to restart Postgres to change the setting of some option. Normally backend options are specified to the postmaster and passed to each backend when it is started. Now they are read from a file.

• we can change options on the fly while a backend is running. We can thus investigate some problem by activating debug messages only when the problem appears. We can also try different values for tunable parameters.

The format of the pg_options file is as follows:

# comment
option=integer_value  # set value for option
option                # set option = 1
option+               # set option = 1
option-               # set option = 0

Note that keyword can also be an abbreviation of the option name defined in backend/utils/misc/trace.c.

The options currently defined in backend/utils/misc/trace.c are the following:

all

Global trace flag. Allowed values are:

0

Trace messages enabled individually

1

Enable all trace messages

-1

Disable all trace messages

verbose

Verbosity flag. Allowed values are:

0

No messages. This is the default.

1

Print information messages.

2

Print more information messages.

query

Query trace flag. Allowed values are:

0

Don't print query.

1

Print a condensed query in one line.

4

Print the full query.

plan

Print query plan.

parse

Print parser output.

rewritten

Print rewritten query.

parserstats

Print parser statistics.

plannerstats

Print planner statistics.

executorstats

Print executor statistics.

shortlocks

Currently unused but needed to enable features in the future.

locks

Trace locks.

userlocks

Trace user locks.

spinlocks

Trace spin locks.

notify

Trace notify functions.

malloc

Currently unused.

palloc

Currently unused.

lock_debug_oidmin

Minimum relation oid traced by locks.

lock_debug_relid

oid, if not zero, of relation traced by locks.

lock_read_priority

Currently unused.

deadlock_timeout

Deadlock check timer.

syslog

syslog flag. Allowed values are:

0

Messages to stdout/stderr.

1

Messages to stdout/stderr and syslog.

2

Messages only to syslog.

hostlookup

Enable hostname lookup in ps_status.

showportnumber

Show port number in ps_status.

notifyunlock

Unlock of pg_listener after notify.

notifyhack

Remove duplicate tuples from pg_listener.

For example my pg_options file contains the following values:

verbose=2
query
hostlookup
showportnumber

Some of the existing code using private variables and option switches has been changed to make use of the pg_options feature, mainly in postgres.c. It would be advisable to modify all existing code in this way, so that we can get rid of many of the switches on the Postgres command line and can have more tunable options with a unique place to put option values.