Re: PostgreSQL Audit Extension

On Wed, Feb 3, 2016 at 9:36 AM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:

> On Wed, Feb 3, 2016 at 10:37 AM, David Steele <david(at)pgmasters(dot)net> wrote:
> > On 2/1/16 11:23 PM, Robert Haas wrote:
>
> >> In
> >> saying that it's arbitrary, I'm not saying it isn't *useful*. I'm
> >> saying there could be five extensions like this that make equally
> >> arbitrary decisions about what to do and how to do it, and they could
> >> all be useful to different people.
> >
> > There *could* be five extensions but there are not. To my knowledge
> > there are two and one is just a more evolved version of the other.
>
> Right now that may be true, although it wouldn't surprise me very much
> to find out that other people have written such extensions and they
> just didn't get as much press. Also, consider the future. It is
> *possible* that your version of pgaudit will turn out to be the be-all
> and the end-all, but it's equally possible that somebody will fork
> your version in turn and evolve it some more. I don't see how you can
> look at the pgaudit facilities you've got here and say that this is
> the last word on auditing and all PostgreSQL users should be content
> with exactly that facility. I find that ridiculous. Look me in the
> eye and tell me that nobody's going to fork your version and evolve it
> a bunch more.
>

​This rings hollow to me. JSON got included with an admittedly weak
feature set and then got significantly improved upon in subsequent
releases. Those who would be incline to fork pgaudit, seeing it already
being in core, would more likely and to the benefit of the community put
that work into improving the existing work.​ My off-the-cuff understanding
is that some current big features (namely the parallel-related stuff) are
also taking this "lets commit smaller but still useful pieces" into core to
build up to this super-feature we want working two years from now. I don't
see any fundamental reason auditing couldn't be given the same opportunity
to improve inside core.

The other major downside of having it in core is that now the feature
release cycle is tied to core. Telling PostGIS they can only release new
features when new versions of PostgreSQL come out would be an unacceptable
situation.

The best of both worlds would be for core to have its own implementation
written as an extension and to readily allow for other implementations to
be plugged in as well. As your alluded to above there are likely a number
of things core really needs to enable such functionality without providing
the entire UI - leaving that for extensions.

> > People who are interested in audit are also understandably leery of
> > downloading code from an untrusted source. Both PGXN and GitHub are The
> > Wild West as far as conservative auditors are concerned.
>
> I hate to be rude here, but that's not my problem. You can put it on
> your corporate web site and let people download it from there. I'm
> sure that auditors are familiar with the idea of downloading software
> from for-profit companies. Do they really not use any software from
> Microsoft or Apple, for example? If the problem is that they will
> trust the PostgreSQL open source project but not YOUR company, then I
> respectfully suggest that you need to establish the necessary
> credibility, not try to piggyback on someone else's.
>

​A bit short-sighted maybe. Endorsing and including such a feature could
open PostgreSQL up to a​ new market being supported by people who right now
are not readily able to join the PostgreSQL community because they cannot
invest the necessary resources to get the horses put before the cart.
Those people, if they could get their clients to more easily use
PostgreSQL, may just find it worth their while to then contribute back to
this new frontier that has been opened up to them. This would ideally
increase the number of contributors and reviewers within the community
which is the main thing that is presently needed.

> > I'll be the first to admit that the design is not the prettiest. Trying
> > to figure out what Postgres is doing internally through a couple of
> > hooks is like trying to replicate the script of a play when all you have
> > is the program. However, so far it has been performed well and been
> > reliable in field tests.
>
> That's good to hear, but again, it's not enough for a core submission.
> Code that goes into our main git repository needs to be "the
> prettiest". I mean it's not all perfect of course, but it should be
> pretty darn good.
>
> Also, understand this: when you get a core submission accepted, the
> core project is then responsible for maintaining that code even if you
> disappear. It's entirely reasonable for the project to demand that
> this isn't going to be too much work. It's entirely reasonable for
> the community to want the design to be very good and the code quality
> to be high.

​So far so good...​

It's entirely reasonable for the community NOT to want to
> privilege one implementation over another.

​This, not so much. At some point if it is decided the feature needs to be
in core then an existing implementation would have to be adopted by core.
If the rule becomes "we cannot commit to core anything that has two
implementations in the wild" becomes a norm then our extension mechanisms
better be damn good. So admitting that you cannot decide is reasonable and
human but choosing not to decide is like abstaining to vote when you are in
a position of appointed authority. You were placed into your position as
committer so that you (along with the others so appointed) could make
decisions like this and as a community member I would hope you would do
everything in your power to inform yourself enough to make such a decision
when asked to do so.

This specific situation does suffer from having multiple different dynamics
to consider - which have already been discussed up-thread - but I guess
fundamentally does the question is: does the community want an official
implementation of auditing present in core?

For the record I would. Ideally I would also have the option to choose a
non-core extension should core's chosen implementation or constraints
(upgrade schedule) not fit my needs. But having a reference
implementation, and thus in-house usage of the various features that an
auditing feature would ideally be able to lean on core to provide, is
appealing. But, as a pre-condition I'd want at least a couple of
committers who whole-heartedly feel the same way and are willing to take on
the necessary research and decisions to make it happen. There are
significant resources out there that will help but that need a leader with
authority to which their efforts can be funneled. That is a personal
opinion that is based more upon the overall structure of the project and
community than it does any specifics of a submitted patch. If there is a
will to get it in then most likely it will happen at some point.

If you don't agree that
> those things are reasonable then we disagree pretty fundamentally on
> the role of the community. The community is a group of people to whom
> I (or you) can give our time and my (or your) code, not a group of
> people who owe me (or you) anything.

​From my point above your acceptance of the commit bit does entail a
certain degree of responsibility to the rest of the community. The good
news is that only those who have already exhibited such a sense of
responsibility are given the privilege. And usually that is sufficient but
in situations where it is not there is also the formal core committee that
can act when the committer group is hamstrung (for lack of a better term)
by its informal nature.

​David J.
​