Re: Auditing extension for PostgreSQL (Take 2)

On Thu, May 7, 2015 at 10:26:55AM -0400, Stephen Frost wrote:
> * Peter Eisentraut (peter_e(at)gmx(dot)net) wrote:
> > On 5/4/15 8:37 PM, Stephen Frost wrote:
> > > I don't follow this logic. The concerns raised above are about changing
> > > our in-core logging. We haven't got in-core auditing and so I don't see
> > > how they apply to it.
> >
> > How is session "auditing" substantially different from statement logging?
>
> David had previously outlined the technical differences between the
> statement logging we have today and what pgAudit does, but I gather
> you're asking about it definitionally, though it ends up amounting to
> much the same to me. Auditing is about "what happened" whereas
> statement logging is "log whatever statement the user sent." pgAudit
> bears this out by logging internal SQL statements and object
> information, unlike what we do with statement logging today.

Well, what I was looking for is how auditing is _conceptually_ different
from logging, e.g. I can clearly explain how authentication (prove who
you are) and authorization (what you are allowed to do) are different.
Your definition above seems to be more behavioral, e.g. what arrived vs.
what happened. It is not clear to me why reporting such information is
conceptually different and requires different infrastructure, i.e. we
could not easily combine authentication and authorization into the same
infrastructure, but logging and auditing seems similar. (Actually,
pg_hba.conf contains authentication and some course-grained
authorization only because it is a good place to do low-overhead
authorization; there was a performance requirement to do it in
pg_hba.conf to prevent DOS attacks.)

> > At least no one has disputed that yet. The only argument against has
> > been that they don't want to touch the logging.
>
> I'm afraid we've been talking past each other here- I'm fully on-board
> with enhancing our in-core logging capabilities and even looking to the
> future at having object auditing included in core. It's not my intent
> to dispute that or to argue against it.
>
> Perhaps I've misunderstood the thrust of this sub-thread, so let me
> explain what I thought the discussion was. My understanding was that
> you were concerned about having session auditing included in pgAudit
> and, further, that you wanted to see our in-core statement logging be
> improved. I agree that we want to improve the in-core statement logging
> and, ideally, have an in-core auditing solution in the future. I was
> attempting to address the concern about having session logging in
> pgAudit by pointing out that it's valuable to have even if our in-core
> statement logging is augmented, and further, having it in pgAudit does
> not preclude or reduce our ability to improve the in-core statement
> logging in the future; indeed, it's my hope that we'll get good feedback
> from users of pgAudit which could guide our in-core implementation. As

What is our history of doing things in contrib because we are not sure
what we want, then moving it into core? My general recollection is that
there is usually something in the contrib version we don't want to add
to core and people are locked into the contrib API, so we are left
supporting it, e.g. xml2, though you could argue that auditing doesn't
have application lock-in and xml2 was tied to an external library
feature.

> for the concern that pgAudit may end up "rotting" in the tree as some
> other contrib modules have, I can say with confidence that we will have
> users of it just as soon as they're able to move to a version of PG
> which includes it and therefore will be supporting it and addressing
> issues as we discover them, as I suspect the others who have been

Uh, why are they not using the PGXN version of pg_audit, and if it is
because it isn't shipped with Postgres, then these seem like unmotivated
users who will complain for some reason if we ever move it out of
contrib.

I guess the over-arching question is whether we have to put this into
contrib so we can get feedback and change the API, or whether using from
PGXN or incrementally adding it to core is the right approach.

> involved in this discussion will be also. Additionally, as discussed
> last summer, we can provide a migration path (which does not need to be
> automated or even feature compatible) from pgAudit to an in-core
> solution and then sunset pgAudit.

Uh, that usually ends badly too.

> Building an in-core solution, in my estimation at least, is going to
> require at least a couple of release cycles and having the feedback from
> users of pgAudit will be very valuable to building a good solution, but
> I don't believe we'll get that feedback without including it.

See above --- is it jump through the user hoops and only then they will
use it and give us feedback? How motivated can they be if they can't
use the PGXN version?

The bottom line is that for the _years_ we ship pg_audit in /contrib, we
will have some logging stuff in postgresql.conf and some in
contrib/pg_audit and that distinction is going to look quite odd. To
the extent you incrementally add to core, you will have duplicate
functionality in both places.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +