Upgrading the backend's error-message infrastructure

(Or, protocol upgrade phase 1...)

After digging through our many past discussions of what to do with error
messages, I have put together the following first-cut proposal. Fire at
will...

Objective
---------

The basic objective here is to divide error reports into multiple
fields, and in particular to include an "error code" field that gives
applications a stable value to test against when they're trying to find
out what went wrong. (I am not spending much space in this proposal on
the question of exactly what set of error codes we ought to have, but
that comes soon.) Peter Eisentraut argued cogently awhile back that the
error codes ought not be hard-wired to specific error message texts,
so this proposal treats them as separate entities.

Wire-protocol changes
---------------------

Error and Notice (maybe also Notify?) msgs will have this structure:

E
x string \0
x string \0
x string \0
\0

where the x's are single-character field identifiers. A frontend should
simply ignore any unrecognized fields. Initially defined fields for Error
and Notice are:

S Severity --- the string is "ERROR", "FATAL", or "PANIC" (if E msg)
or "WARNING", "NOTICE", "DEBUG", "INFO", or "LOG" (if N msg).
(Should this string be localizable? Probably, assuming that the
E/N distinction is all the client library really cares about.)
C Code --- SQLSTATE code for error (a 5-character string per SQL
spec). Not localizable.
M Message --- the string is the primary error message (localized).
D Detail --- secondary error message, carrying more detail about
the problem (localized).
H Hint --- a suggestion what to do about the error (localized).
P Position --- the string is a decimal ASCII integer, indicating
an error cursor position as an index into the original query
string. First character is index 1. Q: measure index in
bytes, or characters? Latter seems preferable considering that
an encoding conversion may have occurred.
F File --- file name of source-code location where error was
reported (__FILE__)
L Line # --- line number of source-code location (__LINE__)
R Routine --- source code routine name reporting error (__func__ or
__FUNCTION__)

S,C,M fields will always appear (at least in Error messages; perhaps
Notices might omit C?). The rest are optional.

Why three textual message fields? 'M' should always appear, 'D' and 'H'
are optional (and relatively rare). The convention is that the primary
'M' message should be accurate but terse (normally one line); if more info
is needed than can reasonably fit on a line, use the detail message to
carry additional lines. A "hint" is something that doesn't directly
describe the error, but is a suggestion what to do to get around it.
'M' and 'D' should be factual, whereas 'H' may contain some guesswork, or
advice that might not always apply. Client interfaces are expected to
report 'M', but might suppress 'D' and/or 'H' depending on factors such as
screen space. (Preferably they should have a verbose mode that shows all
available info, though.)

Error codes
-----------

The SQL spec defines a set of 5-character status codes (called SQLSTATE
values). We'll use these as the language-independent identifiers for
error conditions. There is code space reserved by the spec for
implementation-defined error conditions, which we'll surely need.

Per spec, each of the five characters in a SQLSTATE code must be a digit
'0'-'9' or an upper-case Latin letter 'A'-'Z'. So it's possible to fit a
SQLSTATE code into a 32-bit integer with some simple encoding conventions.
I propose that we use such a representation in the backend; that is,
instead of passing around strings like "1200D" we pass around integers
formed like ((('1' - '0') << 6) + '2' - '0') << 6 ... This should save
a useful amount of space per elog call site, and it won't obscure the code
noticeably since all the common values will be represented as macro names
anyway, something like

#define ERRCODE_DIVISION_BY_ZERO MAKE_SQLSTATE('2','2', '0','1','2')

We need to do some legwork to figure out what set of
implementation-defined error codes we want. It might make sense to look
and see what other DBMSes are using.

Backend source-code representation for extended error messages
--------------------------------------------------------------

How do we generalize the elog() interface to cope with all this stuff?
I don't think I want a function with a fixed parameter list --- some sort
of open-ended API would be a lot more forward-looking. After some fooling
around I've come up with the following proposal.

A typical elog() call might be replaced by

ereport(ERROR, ERRCODE_INTERNAL,
errmsg("Big trouble with table %s", name),
errhint("Bail out now, boss"));

ERROR is the severity level, same as before, and ERRCODE_xxx is (a macro
for) the appropriate SQLSTATE code. The rest is a variable-length list of
optional items, each expressed as a subsidiary function call. This
representation preserves the single-function-call appearance of elog()
calls, which is convenient for coding purposes, but it gives us something
akin to labeled optional parameters instead of C's usual fixed parameter
list.

How does this work, exactly? Well, errmsg() and errhint() are indeed
functions, but ereport is actually a macro:

#define ereport errstart(__FILE__, __LINE__, __FUNCTION__), errfinish

(__FUNCTION__ is only used if we are compiling in gcc). errstart() pushes
an empty entry onto an error-data-collection stack and fills in the
behind-the-scenes file/line entries. errmsg() and friends stash values
into the top-level stack entry. Finally errfinish() assembles and emits
the completed message, then pops the stack. By using a stack, we can be
assured that things will work correctly if a message is logged by some
subroutine called in the parameters to ereport (not too unlikely when you
think about formatting functions like format_type_be()).

Behind the scenes we have

extern void errstart(const char *filename, int lineno, const char *funcname);
extern void errfinish(int elevel, int sqlerrorcode, ...);

The individual routines for adding optional items to the error report are:

extern int errmsg(const char *fmt, ...);

Primary error message, possibly with parameters interpolated per the
existing elog conventions (sprintf-like format string). The first
parameter is gettext-ified. Primary messages should be one line if at
all possible (make it complete but succinct).

extern int errdetail(const char *fmt, ...);

Adds an optional secondary error message, for use when not all the
description of an error condition can be fit into a reasonably terse
primary error message. Functionality essentially the same as errmsg().
errdetail output can run to multiple lines, but bear in mind that some
client APIs may not show it.

extern int errhint(const char *fmt, ...);

Adds a "hint"; behavior otherwise similar to errdetail().

An example is that the existing
elog(ERROR, "Adding columns with defaults is not implemented."
"\n\tAdd the column, then use ALTER TABLE SET DEFAULT.");
becomes
ereport(ERROR, ERRCODE_something,
errmsg("Adding columns with defaults is not implemented"),
errhint("Add the column, then use ALTER TABLE SET DEFAULT"));
Notice that we got rid of a hard-wired decision about presentation layout.

extern int errmsg_internal(const char *fmt, ...);

Like errmsg() except that the first parameter is not subject to
gettext-ification. My thought is that this would be used for internal
can't-happen conditions; there's no need to make translators labor over
translating stuff like "eval_const_expressions: unexpected boolop %d",
nor even to make them think about whether they need to. The only part
of such a message that needs internationalization is the hint "Please
report this problem to pgsql-bugs", which should be added automatically
by errmsg_internal(). The ERRCODE should almost always be "internal
error" if this is used.

extern int errfunction(const char *funcname);

Provides the name of the function reporting the error. In gcc-compiled
backends, the function name will be provided automatically by errstart,
but there will be some places where we need the name to be available even
in a non-gcc build. My thought is that
elog(WARNING, "PerformPortalFetch: portal \"%s\" not found",
stmt->portalname);
becomes
ereport(WARNING, ERRCODE_INVALID_CURSOR_NAME,
errmsg("portal \"%s\" not found", stmt->portalname),
errfunction("PerformPortalFetch"));
This gets us out of the habit of including function name in the primary
error message, while still leaving enough info that we can construct
a backwards-compatible error report for old clients. (I'm thinking that
if errfunction() is present, the function name and a colon would be
prepended to the primary error message, but only if sending to an
old-protocol client.)

extern int errposition(int cursorpos);

Provides error position info (an offset into the original query text).
For the moment this is probably only going to happen for scanner and
grammar errors.

NOTE: a variant scheme would treat the SQLSTATE code as an optional
parameter too, ie you'd write
ereport(ERROR, errcode(ERRCODE_xxx), ...
This would just be excess verbiage if most or all ereport calls specify
error codes --- but for the errmsg_internal case we could leave out
errcode(), expecting it to default to "internal error". Any thoughts on
which way is better?

Backwards compatibility
-----------------------

When talking to an old-protocol client, the ereport code will assemble the
appropriate elements of the available data to produce an approximately
backward-compatible message, that is, ye olde
ERROR: routine: primary message
(where routine: appears only if errfunction() was called).

elog() will remain available for at least a couple of releases, so as not
to force immediate updates of user-written extension functions. It will
default to some implementation-defined SQLSTATE value for "unspecified
error". We can change "elog" into a macro similar to "ereport" so that we
can get file/line number info. (This means we're only giving source-code
not object-code compatibility with extension functions, but that's
generally the case anyway during PG major version updates.)

Comments?

regards, tom lane