Re: Logging: stderr vs syslog?

rsyslog on CentOS7, no central server, just local log files.

I appreciate your suggestion of going to syslog support but I'm more
interested in how those in the pgsql-admin community handle these concerns.
I'm not asking for syslog support.

Don.

On Sat, Aug 5, 2017 at 7:31 PM, Evan Rempel <erempel(at)uvic(dot)ca> wrote:

> Tou are now asking questions that are more appropriate for the specific
> syslog product you are using.
> rsyslog, syslogd, syslog-ng etc.
>
> In general you need to ask yourself what the ramifications of loosing the
> most recent locally stored syslog messages. If you are forwarding you
> syslog message to a central syslog server, or perhaps event two syslog
> servers then the loss of some local syslog messages in the event of a local
> host crash is not very significant.
>
> If you can live with the local loss then adding a "no sync" option to your
> syslog configuration is a good idea.
>
> For further details please follow up with your syslog specific support
> channel.
>
> Evan.
>
>
> On 08/04/2017 10:22 PM, Don Seiler wrote:
>
> Thanks everyone for your feedback. I've got a test setup running with
> syslog and logrotate and it's working well.
>
> Is there any concern with the syslog file sync where it's suggested to put
> a dash before the file name to remove the sync requirement?
>
> Don.
>
> On Aug 4, 2017 2:51 PM, "Evan Rempel" <erempel(at)uvic(dot)ca> wrote:
>
>> I have been a systems administrator of PostgreSQL since version 7.0. I am
>> also a primary logging architect for a 4000+ device network infrastructure
>> and SEIM alerting infrastructure architect.
>>
>> Syslog is by far the best approach to logging. With the right product it
>> is way FASTER than stderr and there are a ton of tools to parse, analyze,
>> view and report on syslog streams.
>>
>> There is one caveat with PostgreSQL version <= 9.5 and that is that
>> syslog messages are wrapped at approx 80 characters which makes parsing and
>> error detection problematic. pgBadger may address this limitation by
>> unwrapping such logs messages whereas generic log parsing engines do not
>> have any specialized knowledge about how PostgreSQL lines might be wrapped.
>> PostgreSQL version <= 9.5 this is a compile time only option but starting
>> in 9.6 this is a runtime configuration directive.
>>
>> There are two strong reasons for using syslog.
>>
>> 1. In a well architected logging solution the syslog process on the host
>> will also send the log messages to a central log server. This means that if
>> the database server is compromised leading to an untrusted set of log
>> files, there is a trusted copy of the logs on another server.
>>
>> 2. When running a high availability or clustered database all of the logs
>> can be aggregated to a central log server which places all of the logs from
>> all of the database servers into one easy to read/parse/process location.
>>
>> I hope this provides some rationale for using syslog.
>>
>> Evan.
>>
>> On 08/04/2017 09:26 AM, Don Seiler wrote:
>>
>> I've just inherited a few PostgreSQL DBs, having come from Oracle land.
>> I'm looking to shore up the logging situation. Right now we use stderr
>> logging and they get rotated based on size threshold. I'd like for those
>> old logs to be gzipped so we can keep more on disk rather than current
>> method of just deleting old logs to free up space. This is mostly on pgsql
>> 9.2 with a couple of 9.3, but I'm planning to upgrade everything to 9.6.3
>> when I get my feet on solid ground.
>>
>> Couple of question around this:
>>
>> 1. I thought logrotate would be a no-brainer here, but it sounds like
>> I should then change to use syslog rather than stderr. I've read some
>> caveats around syslog needing to sync files and potentially slow things
>> down. I'm wondering if any grizzled production postgres veterans could
>> offer up their experience.
>> 2. Alternatively I could just keep it going with stderr and have a
>> separate script run find/gzip on log files beyond a certain mtime
>> threshold. This would probably be the quickest to implement, but I'd much
>> rather use logrotate facilities if there are no strong opinions against
>> using syslog.
>>
>> Thanks in advance for your time, I'm sure I'll be making a lot of us of
>> these mailing lists in the not-too-distant future.
>>
>> Don.
>>
>> --
>> Don Seiler
>> www.seiler.us
>>
>>
>>
>

--
Don Seiler
www.seiler.us