Re: tap tests driving the database via psql

On Tue, 30 Jul 2019 at 21:40, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:

> Craig Ringer <craig(at)2ndquadrant(dot)com> writes:
> > On Sun, 28 Jul 2019 at 03:15, Andres Freund <andres(at)anarazel(dot)de> wrote:
> >> 1) Just depend on DBD::Pg being installed. It's fairly common, after
> >> all. It'd be somewhat annoying that we'd often end up using a
> >> different version of libpq than what we're testing against. But in
> >> most cases that'd not be particularly problematic.
>
> > I advocated for this in the past, and still think it's the best option.
>
> I think the not-same-libpq issue would be a much bigger problem than either
> of you are accounting for.

OK. So rather than building our own $everything from scratch, lets look at
solving that. I'd argue that the PostgreSQL community has enough work to do
maintaining the drivers that are fairly directly community-owned like
PgJDBC and psqlODBC, let alone writing new ones for less-in-demand
languages just for test use.

Perl modules support local installations. Would it be reasonable to require
DBI to be present, then rebuild DBD::Pg against our libpq during our own
test infra compilation, and run it with a suitable LD_LIBRARY_PATH or using
rpath? That'd actually get us a side-benefit of some test coverage of
DBD::Pg and libpq.

Note that I'm not saying this is the only or right way. Your concerns about
using a system libpq are entirely valid, as are your concerns about trying
to link to our libpq using a DBD::Pg originally built against the system
libpq. I've just been dealing with issues similar to those today and I know
how much of a hassle it can be. However, I don't think "it's hard" is a
good reason to write a whole new driver and potentially do a
Netscape/Mozilla.

> * Since we'd have to presume a possibly-very-back-rev libpq, we could
> never add tests related to any recent libpq bug fixes or new features.
>

We can feature-test. We're not dealing with pg_regress where it's
essentially impossible to do anything conditionally without blocks of
plpgsql. We're dealing with Perl and Test::More where we can make tests
conditional, we can probe the runtime environment to decide whether we can
run a given test, etc.

Also, lets compare to the status quo. Will it be worse than exec()ing psql
with a string argument and trying to do sane database access via stdio?
Definitely. Not.

> * The system libpq might have a different idea of default port
> and/or socket directory than the test build. Yeah, we could work
> around that by forcing the choice all the time, but it would be a
> constant hazard.
>

I'd call that a minor irritation personally, as all we have to do is set up
the environment and we're done.

> * We don't have control over what else gets brought in beside libpq.
>

That's a very significant point that we must pay attention to.

Anyone who's worked with nss will probably know the pain of surprise
library conflicts arising from nss plugins being loaded unexpectedly into
the program. I still twitch thinking about libnss-ldap.

It'd make a lot of sense to capture "ldd" output and/or do a minimal run
with LD_DEBUG set during buildfarm test runs to help identify any
interesting surprises.

> Depending on the whims of the platform packagers, there might need
> to be other parts of the platform's default postgres installation,
> eg psql, sitting in one's PATH. Again, this wouldn't be too much
> of a hazard for pre-debugged test scripts --- but it would be a huge
> hazard for developers, who do lots of things manually and would always be
> at risk of invoking the wrong psql. I learned long ago not to have any
> part of a platform's postgres packages in place on my development systems,
> and I will fight hard against any test procedure change that requires me
> to do differently.
>

TBH I think that's a habit/procedure issue. That doesn't make it invalid,
but it might not be as big a concern for everyone as for you. I have a
shell alias that sets up my environment and another that prints the current
setup. I never run into issues like this, despite often multi-tasking while
tired. Not only that, I find platform postgres extremely useful to have on
my systems to use for simple cross-version tests.

If we adopt something like I suggested above where we (re)build DBD::Pg
against our installed pg and libpq, that wouldn't be much of an issue
anyway. We'd want a nice way to set up the runtime environment in a shell
for manual testing of course, like a generated .sh we can source. But
frankly I'd find that a useful thing to have in postgres anyway. I can't
count the number of times I've wished there was an easy way to pause
pg_regress and launch a psql session against its temp instance, or do the
same with a TAP test.

Now, none of these things are really a problem with DBD/DBI as such
> --- rather, they are reasons not to depend on a pre-packaged build
> of DBD::Pg that depends on a pre-packaged build of libpq.so.
>

Yeah. While I don't agree with your conclusion in terms of how you weigh
the priorities, I agree that all your points are valid and make sense. I'd
personally go ahead anyway, because the status quo is pretty bad and an
incremental improvement would still be significant. But I understand your
reluctance a bit better.

> I haven't looked at the size, or the license, of DBD::Pg ... but
> could it be sane to include our own modified copy in the tree?
>

Or ... clone it on demand? I won't suggest git submodules, because they're
awkward bodgy hacks, but we can reasonably enough just fetch the sources
when we need them so long as they aren't a hard dependency on building
postgres. Which they wouldn't need to be; at worst we'd have to skip the
TAP tests if no DBD::Pg was found.

We could also permit falling back to system DBD::Pg with a suitable
visible message in logs to ensure nobody gets confused.

I'd very strongly prefer not to have a copy in-tree polluting the postgres
history and generally creating a mess. We don't even have "blessed" drivers
that live on community infra like psqlODBC or PgJDBC in-tree.

> (Forking DBD::Pg would also let us add custom testing features
> to it ...)
>

Lets not. Unless utterly unavoidable. There's enough to do already.

> > The community IMO wastes *so* much time on not-invented-here make-work
> and
> > on jumping through hoops to avoid depending on anything newer than the
> late
> > '90s.
>
> That's an unnecessary, and false, ad-hominem attack.
>

It's none of the above. It's an honest opinion, not an attack, and it's not
directed at you or any one other person.

--
Craig Ringer http://www.2ndQuadrant.com/
2ndQuadrant - PostgreSQL Solutions for the Enterprise