Re: backup manifests

Hi,

There is one scenario  where  i somehow able to run pg_validatebackup
successfully but when i tried to start the server , it is failing

Steps to reproduce -
--create 2 base backup directory
[centos(at)tushar-ldap-docker bin]$ ./pg_basebackup -D db1
[centos(at)tushar-ldap-docker bin]$ ./pg_basebackup -D db2

--run pg_validatebackup , use backup_manifest of db1 directory against 
db2/  . Will get an error
[centos(at)tushar-ldap-docker bin]$ ./pg_validatebackup -m
db1/backup_manifest db2/
pg_validatebackup: * manifest_checksum =
5b131aff4a4f86e2a53efd84b003a67b9f615decb0039f19033eefa6f43c1ede
pg_validatebackup: error: checksum mismatch for file "backup_label"
--copy the backup_level of db1 to db2 folder
[centos(at)tushar-ldap-docker bin]$ cp db1/backup_label db2/.

--run pg_validatebackup .. working fine
[centos(at)tushar-ldap-docker bin]$ ./pg_validatebackup -m
db1/backup_manifest db2/
pg_validatebackup: * manifest_checksum =
5b131aff4a4f86e2a53efd84b003a67b9f615decb0039f19033eefa6f43c1ede
pg_validatebackup: backup successfully verified
[centos(at)tushar-ldap-docker bin]$

--try to start the server
[centos(at)tushar-ldap-docker bin]$ ./pg_ctl -D db2 start -o '-p 7777'
waiting for server to start....2020-03-05 15:33:53.471 IST [24049] LOG: 
starting PostgreSQL 13devel on x86_64-pc-linux-gnu, compiled by gcc
(GCC) 4.8.5 20150623 (Red Hat 4.8.5-39), 64-bit
2020-03-05 15:33:53.471 IST [24049] LOG:  listening on IPv6 address
"::1", port 7777
2020-03-05 15:33:53.471 IST [24049] LOG:  listening on IPv4 address
"127.0.0.1", port 7777
2020-03-05 15:33:53.473 IST [24049] LOG:  listening on Unix socket
"/tmp/.s.PGSQL.7777"
2020-03-05 15:33:53.476 IST [24050] LOG:  database system was
interrupted; last known up at 2020-03-05 15:32:51 IST
2020-03-05 15:33:53.573 IST [24050] LOG:  invalid checkpoint record
2020-03-05 15:33:53.573 IST [24050] FATAL:  could not locate required
checkpoint record
2020-03-05 15:33:53.573 IST [24050] HINT:  If you are restoring from a
backup, touch
"/home/centos/pg13_bk_mani/edb/edbpsql/bin/db2/recovery.signal" and add
required recovery options.
    If you are not restoring from a backup, try removing the file
"/home/centos/pg13_bk_mani/edb/edbpsql/bin/db2/backup_label".
    Be careful: removing
"/home/centos/pg13_bk_mani/edb/edbpsql/bin/db2/backup_label" will result
in a corrupt cluster if restoring from a backup.
2020-03-05 15:33:53.574 IST [24049] LOG:  startup process (PID 24050)
exited with exit code 1
2020-03-05 15:33:53.574 IST [24049] LOG:  aborting startup due to
startup process failure
2020-03-05 15:33:53.575 IST [24049] LOG:  database system is shut down
 stopped waiting
pg_ctl: could not start server
Examine the log output.
[centos(at)tushar-ldap-docker bin]$

regards,

On 3/5/20 1:09 PM, Rajkumar Raghuwanshi wrote:
> Hi,
>
> In a negative test scenario, if I changed size to -1 in
> backup_manifest, pg_validatebackup giving
> error with a random size number.
>
> [edb(at)localhost bin]$ ./pg_basebackup -p 5551 -D /tmp/bold
> --manifest-checksum 'SHA256'
> [edb(at)localhost bin]$ ./pg_validatebackup /tmp/bold
> pg_validatebackup: backup successfully verified
>
> --change a file size to -1 and generate new checksum.
> [edb(at)localhost bin]$ vi /tmp/bold/backup_manifest
> [edb(at)localhost bin]$ shasum -a256 /tmp/bold/backup_manifest
> c3d7838cbbf991c6108f9c1ab78f673c20d8073114500f14da6ed07ede2dc44a
>  /tmp/bold/backup_manifest
> [edb(at)localhost bin]$ vi /tmp/bold/backup_manifest
>
> [edb(at)localhost bin]$ ./pg_validatebackup /tmp/bold
> pg_validatebackup: error: "global/4183" has size 0 on disk but size
> *18446744073709551615* in the manifest
>
> Thanks & Regards,
> Rajkumar Raghuwanshi
>
>
> On Thu, Mar 5, 2020 at 9:37 AM Suraj Kharage
> <suraj(dot)kharage(at)enterprisedb(dot)com
> <mailto:suraj(dot)kharage(at)enterprisedb(dot)com>> wrote:
>
>
> On Wed, Mar 4, 2020 at 7:21 PM tushar
> <tushar(dot)ahuja(at)enterprisedb(dot)com
> <mailto:tushar(dot)ahuja(at)enterprisedb(dot)com>> wrote:
>
> Hi,
>
> There is a scenario in which i add something inside the
> pg_tablespace directory , i am getting an error like-
>
> pg_validatebackup: * manifest_checksum =
> 77ddacb4e7e02e2b880792a19a3adf09266dd88553dd15cfd0c22caee7d9cc04
> pg_validatebackup: error:
> "pg_tblspc/16385/*PG_13_202002271*/test" is present on disk
> but not in the manifest
>
> but if i remove 'PG_13_202002271 ' directory then there is no
> error
>
> [centos(at)tushar-ldap-docker bin]$ ./pg_validatebackup data
> pg_validatebackup: * manifest_checksum =
> 77ddacb4e7e02e2b880792a19a3adf09266dd88553dd15cfd0c22caee7d9cc04
> pg_validatebackup: backup successfully verified
>
>
> This seems expected considering current design as we don't log the
> directory entries in backup_manifest. In your case, you have
> tablespace with no objects (empty tablespace) then backup_manifest
> does not have any entry for this hence when you remove this
> tablespace directory, validator could not detect it.
>
> We can either document it or add the entry for directories in the
> manifest. Robert may have a better idea on this.
>
> --
> --
>
> Thanks & Regards,
> Suraj kharage,
> EnterpriseDB Corporation,
> The Postgres Database Company.
>

--
regards,tushar
EnterpriseDB https://www.enterprisedb.com/
The Enterprise PostgreSQL Company