Re: WIP patch - INSERT-able log statements

On Sat, 17 Feb 2007, FAST PostgreSQL wrote:

> #log_output_type = 'text' #Valid values are 'SQL' or 'text'
> Defaults to 'text' which is status quo. If it is set to 'SQL' log will
> be output as INSERT commands.

This assumes someone wants either the INSERT-able logs or the current,
easily readable ones. I know I don't want either--I want both. There are
times I want to look through the logs with a text editor, there are times
where I want to query against them.

I would suggest treating this similarly to how the Windows eventlog is
handled: made SQL INSERT format another option available to
log_destination, so it can be combined with the existing formats. In
addition to the syslog concerns you already mentioned (which are
themselves a showstopper for using this feature in companies that rely on
or aggregate syslogs), I know I'd want to keep the existing logs rolling
in parallel while I tested out the SQL-based version for a while, before
cutting over to exclusively INSERT format logs.

I've thought a bit about how to implement this TODO already (I have a log
file parser and I hate maintaining it), and the only thing that made sense
to me was giving a new parameter with the filename to output to in this
format. For example, make a new log_sql_filename with the same syntax
already used for log_filename. There will probably need to be a second
parameter for the table name to insert into as you've already commented
on. And like Joshua has already suggested, the main useful applications
for this feature I've thought of all involve reading from the INSERT-able
logs in real-time, using something like "tail -f", and pumping that data
immediately into a logger table.

Also, I feel that supporting the whole log_line_prefix syntax for this
feature is not just overkill, it's a bad idea. Output everything in a
standard, complete format instead, and then it becomes easy for the
community at large to build tools on top of that to analyze the log
database entries instead of having so many ad-hoc approaches. You want a
subset, use a view or copy just the fields you want into another table.
I would guess this simplifies the patch as well.

--
* Greg Smith gsmith(at)gregsmith(dot)com http://www.gregsmith.com Baltimore, MD