Re: log_statement and syslog severity

On Wed, Mar 10, 2010 at 8:51 AM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> Greg Sabino Mullane wrote:
>> Bruce replied:
>> ...
>> >> This means that, even using syslog as a destination, it's not possible for
>> >> me to filter statements without some sort of log-text parsing, which I'd
>> >> prefer to avoid on effort, performance and data-integrity grounds.
>>
>> > Our logging system is very flexible, but not work-free on the user end.
>> > I don't see us changing things in that area.
>>
>> Bruce, that's a little harsh, I think the original poster has a legitimate
>> request. Personally, I'd love to be able to split the logs on various things,
>> the most important to me being durations and per-database. I looked at the
>> code about a year ago to see how hard this would be and found it non-trivial
>> (for me), as we're really assuming hard things go to a single filehandle.
>> It's definitely an area for improvement, and should be a TODO if not already.
>
> This issue has been discussed and I think the community conclusion was
> that this should not be done by the database but rather by external
> tools.  I think I was giving an accurate portrayal of the odds of this
> getting added.  I do not think there is enough support for this to be a
> TODO item.

How do you plug in this external tool?

Installing a filter on stderr doesn't play well with packaged versions of PostgreSQL and probably not Windows either. You also don't get easily machine readable output.

It might be possible to trick csvlog to log to a static filename, and perhaps substituting that with a named pipe might work (under unix at least).

syslog doesn't give you easily machine readable output. I'm not sure how syslog implementations handle high load (our sysadmins won't use it, so I haven't investigated this further).

I need to be analyzing log messages from PostgreSQL in real time, so am starting to investigate solutions. It seems painful, which would be avoidable for future generations if PostgreSQL could spawn a subprocess and send log messages to that in a machine readable format. Perhaps useful filters might start to exist and eventually end up in contrib or core.

--
Stuart Bishop <stuart(at)stuartbishop(dot)net>
http://www.stuartbishop.net/