Re: Similar to csvlog but not really, json logs?

* Tom Lane (tgl(at)sss(dot)pgh(dot)pa(dot)us) wrote:
> I wrote:
> > Stephen Frost <sfrost(at)snowman(dot)net> writes:
> >> Consider an audit system where which columns end up in the audit log are
> >> controlled by issuing ALTER TABLE .. ALTER COLUMN type statements.
>
> > I'd like to consider such a thing, but it seems like utter pie in the
> > sky.
>
> On further thought: the existing postmaster log is primarily an error log,
> and for the reasons I mentioned, it seems useless to imagine filtering it
> on security-based grounds. You can either route it to /dev/null or
> restrict viewing to suitably privileged people; there is no middle ground
> with any usefulness.

Well, see my other email about possibly making it useful while also not
leaking information. If that's not possible (and that may well be the
case), then so be it and it'll be addressed by suitably privileged
individuals, but I've not quite given up on it entirely yet. Still, I
agree that it's not quite the same as auditing.

> However: for some values of "audit" it seems like an audit log could
> consist of reports of changes actually applied to the database. If that's
> your definition then it's surely possible to imagine omitting particular
> columns (or rows) from the output, because we've already eliminated all
> the cases where the system couldn't make sense of the input it was fed.
> So I think Alvaro was right to suggest that maybe the logical-decoding
> subsystem, rather than the existing logging subsystem, is where to look
> for solutions. You could do logical decoding of changes and emit a
> filtered version of that to some output that's independent of the current
> postmaster logging arrangements.

That's definitely one avenue we've looked at- but it also has certain
limitations (can you figure out which role on which connection caused
the record in the logical decoding system?). I believe Andreas has
ideas about how to improve that situation and that may end up being the
correct answer, though it does leave unaddressed a wide area of "we
parsed the command- knew exactly what you intended to do, but we didn't
allow you to do it" type of scenarios (eg: permission denied) that
auditors are often very interested in.

Also, just to clarify, we're certainly not making promises that these
things will suddenly appear- tomorrow's internal meeting is primairly
with a certain Greg Smith to discuss the requirements and try to flesh
out ideas to propose to the community and get feedback on.

Thanks!

Stephen