Re: backup manifests

From: David Steele <david(at)pgmasters(dot)net>
To: Andres Freund <andres(at)anarazel(dot)de>, Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: Noah Misch <noah(at)leadboat(dot)com>, Stephen Frost <sfrost(at)snowman(dot)net>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Suraj Kharage <suraj(dot)kharage(at)enterprisedb(dot)com>, tushar <tushar(dot)ahuja(at)enterprisedb(dot)com>, Rajkumar Raghuwanshi <rajkumar(dot)raghuwanshi(at)enterprisedb(dot)com>, Rushabh Lathia <rushabh(dot)lathia(at)gmail(dot)com>, Tels <nospam-pg-abuse(at)bloodgate(dot)com>, Andrew Dunstan <andrew(dot)dunstan(at)2ndquadrant(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>, Jeevan Chalke <jeevan(dot)chalke(at)enterprisedb(dot)com>, vignesh C <vignesh21(at)gmail(dot)com>
Subject: Re: backup manifests
Date: 2020-03-30 22:56:58
Message-ID: f7a6d5b0-6849-d0c7-f4de-fb6bd3a1777c@pgmasters.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 3/30/20 5:08 PM, Andres Freund wrote:
>
> The data in the backup label isn't sufficient though. Without having
> parsed the timeline file there's no way to verify that the correct WAL
> is present. I guess we can also add client side tools to parse
> timelines, add command the fetch all of the required files, and then
> interpret that somehow.
>
> But that seems much more complicated.
>
> Imo it makes sense to want to be able verify that WAL looks correct even
> transporting WAL using another method (say archiving) and thus using
> pg_basebackup's -Xnone.
>
> For the manifest to actually list what's required for the base backup
> doesn't seem redundant to me. Imo it makes the manifest file make a good
> bit more sense, since afterwards it actually describes the whole base
> backup.

FWIW, pgBackRest stores the backup WAL stop/start in the manifest. To
get this information after the backup is complete requires parsing the
.backup file which doesn't get stored in the backup directory by
pg_basebackup. As far as I know, this is only accessibly to solutions
that implement archive_command. So, pgBackRest could do that but it
seems far more trouble than it is worth.

Regards,
--
-David
david(at)pgmasters(dot)net

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2020-03-30 22:58:20 Re: fix for BUG #3720: wrong results at using ltree
Previous Message James Coleman 2020-03-30 22:53:47 Re: [PATCH] Incremental sort (was: PoC: Partial sort)