Re: pgsql: Validate page level checksums in base backups

On Wed, Apr 4, 2018 at 8:22 PM, Michael Banck <michael(dot)banck(at)credativ(dot)de>
wrote:

> Hi,
>
> On Wed, Apr 04, 2018 at 07:25:11PM +0200, Magnus Hagander wrote:
> > On Wed, Apr 4, 2018 at 3:47 PM, Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>
> > wrote:
> > > Michael Banck wrote:
> > >
> > > > However, the pg_basebackup testsuite takes up 800+ MB to run,
> > >
> > > Uh, you're right. This seems a bit over the top. Can we reduce that
> > > without losing coverage? We've gone great lengths to avoid large
> > > amounts of data in tests elsewhere.
> >
> > That didn't come out of this patch, right? This is a pre-existing issue?
>
> It was around that ballpark before, but this patch probably made
> things worse as it adds four additional datadirs (around 40 MB each
> here) and we are close to 1 GB now.
>
> I haven't looked at the other testsuites, but if it is ok to cleanup the
> basebackups after each set of tests suceeded, that would alleviate the
> problem.
>
> Otherwise, I had a quick look and there is no obvious outlier; the
> pgdata is 220 MB after the testrun (195 MB of which is WAL, maybe that
> could be cut down somehow?) and the base backups are 22-40 MB each, and
> there is around 20 of them, so that adds up to more than 750 MB.
>
>
It certainly seems reasonable to delete the base backups once they're made,
after each step, rather than keeping them around forever.

Do we have a precedent somewhere for how we do this, or does our test
framework already have a way to do it? How are all the actual data
directories etc cleaned up?

Or should it just be a matter of sprinkling some unlink() calls throughout
the test file?

--
Magnus Hagander
Me: https://www.hagander.net/ <http://www.hagander.net/>
Work: https://www.redpill-linpro.com/ <http://www.redpill-linpro.com/>