Re: backup manifests

On Sat, Mar 21, 2020 at 4:00 AM Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>
> On Mon, Mar 16, 2020 at 2:03 AM Suraj Kharage
> <suraj(dot)kharage(at)enterprisedb(dot)com> wrote:
> > One more suggestion, recent commit (1933ae62) has added the PostgreSQL home page to --help output.
>
> Good catch. Fixed. I also attempted to address the compiler warning
> you mentioned in your other email.
>
> Also, I realized that the previous patch versions didn't handle the
> hex-encoded path format that we need to use for non-UTF8 filenames,
> and that there was no easy way to test that format. So, in this
> version I added an option to force all pathnames to be encoded in that
> format. I also made that option capable of suppressing the backup
> manifest altogether. Other than that, this version is pretty much the
> same as the last version, except for a few additional test cases which
> I added to get the code coverage up even a little more. It would be
> nice if someone could test whether the tests pass on Windows.
>

On my CentOS, the patch gives below compilation failure:
pg_validatebackup.c: In function ‘parse_manifest_file’:
pg_validatebackup.c:335:19: error: assignment left-hand side might be
a candidate for a format attribute [-Werror=suggest-attribute=format]
context.error_cb = report_manifest_error;

I have tested it on Windows and found there are multiple failures.
The failures are as below:
Test Summary Report
---------------------------------------
t/002_algorithm.pl (Wstat: 512 Tests: 5 Failed: 4)
Failed tests: 2-5
Non-zero exit status: 2
Parse errors: Bad plan. You planned 19 tests but ran 5.
t/003_corruption.pl (Wstat: 256 Tests: 14 Failed: 7)
Failed tests: 2, 4, 6, 8, 10, 12, 14
Non-zero exit status: 1
Parse errors: Bad plan. You planned 44 tests but ran 14.
t/004_options.pl (Wstat: 4352 Tests: 25 Failed: 17)
Failed tests: 2, 4, 6-12, 14-17, 19-20, 22, 25
Non-zero exit status: 17
t/005_bad_manifest.pl (Wstat: 1792 Tests: 44 Failed: 7)
Failed tests: 18, 24, 26, 30, 32, 34, 36
Non-zero exit status: 7
Files=6, Tests=109, 72 wallclock secs ( 0.05 usr + 0.01 sys = 0.06 CPU)
Result: FAIL

Failure Report
------------------------
t/002_algorithm.pl ..... 1/19
# Failed test 'backup ok with algorithm "none"'
# at t/002_algorithm.pl line 33.

# Failed test 'backup manifest exists'
# at t/002_algorithm.pl line 39.

t/002_algorithm.pl ..... 4/19 # Failed test 'validate backup with
algorithm "none"'
# at t/002_algorithm.pl line 53.

# Failed test 'backup ok with algorithm "crc32c"'
# at t/002_algorithm.pl line 33.
# Looks like you planned 19 tests but ran 5.
# Looks like you failed 4 tests of 5 run.
# Looks like your test exited with 2 just after 5.
t/002_algorithm.pl ..... Dubious, test returned 2 (wstat 512, 0x200)
Failed 18/19 subtests
t/003_corruption.pl .... 1/44
# Failed test 'intact backup validated'
# at t/003_corruption.pl line 110.

# Failed test 'corrupt backup fails validation: extra_file: matches'
# at t/003_corruption.pl line 117.
# 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
# doesn't match '(?^:extra_file.*present on disk but not in the manifest)'
t/003_corruption.pl .... 5/44
# Failed test 'intact backup validated'
# at t/003_corruption.pl line 110.
t/003_corruption.pl .... 7/44
# Failed test 'corrupt backup fails validation:
extra_tablespace_file: matches'
# at t/003_corruption.pl line 117.
# 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
# doesn't match '(?^:extra_ts_file.*present on disk but not in the
manifest)'
t/003_corruption.pl .... 9/44
# Failed test 'intact backup validated'
# at t/003_corruption.pl line 110.

# Failed test 'corrupt backup fails validation: missing_file: matches'
# at t/003_corruption.pl line 117.
# 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
# doesn't match '(?^:pg_xact/0000.*present in the manifest but not on disk)'
t/003_corruption.pl .... 13/44
# Failed test 'intact backup validated'
# at t/003_corruption.pl line 110.
# Looks like you planned 44 tests but ran 14.
# Looks like you failed 7 tests of 14 run.
# Looks like your test exited with 1 just after 14.
t/003_corruption.pl .... Dubious, test returned 1 (wstat 256, 0x100)
Failed 37/44 subtests
t/004_options.pl ....... 1/25
# Failed test '-q succeeds: exit code 0'
# at t/004_options.pl line 25.

# Failed test '-q succeeds: no stderr'
# at t/004_options.pl line 27.
# got: 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
# expected: ''

# Failed test '-q checksum mismatch: matches'
# at t/004_options.pl line 37.
# 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
# doesn't match '(?^:checksum mismatch for file \"PG_VERSION\")'
t/004_options.pl ....... 7/25
# Failed test '-s skips checksumming: exit code 0'
# at t/004_options.pl line 43.

# Failed test '-s skips checksumming: no stderr'
# at t/004_options.pl line 43.
# got: 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
# expected: ''

# Failed test '-s skips checksumming: matches'
# at t/004_options.pl line 43.
# ''
# doesn't match '(?^:backup successfully verified)'

# Failed test '-i ignores problem file: exit code 0'
# at t/004_options.pl line 48.

# Failed test '-i ignores problem file: no stderr'
# at t/004_options.pl line 48.
# got: 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
# expected: ''

# Failed test '-i ignores problem file: matches'
# at t/004_options.pl line 48.
# ''
# doesn't match '(?^:backup successfully verified)'

# Failed test '-i does not ignore all problems: matches'
# at t/004_options.pl line 57.
# 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
# doesn't match '(?^:pg_xact.*is present in the manifest but not on disk)'

# Failed test 'multiple -i options work: exit code 0'
# at t/004_options.pl line 62.

# Failed test 'multiple -i options work: no stderr'
# at t/004_options.pl line 62.
# got: 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
# expected: ''

# Failed test 'multiple -i options work: matches'
# at t/004_options.pl line 62.
# ''
# doesn't match '(?^:backup successfully verified)'

# Failed test 'multiple problems: missing files reported'
# at t/004_options.pl line 71.
# 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
# doesn't match '(?^:pg_xact.*is present in the manifest but not on disk)'

# Failed test 'multiple problems: checksum mismatch reported'
# at t/004_options.pl line 73.
# 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
# doesn't match '(?^:checksum mismatch for file \"PG_VERSION\")'

# Failed test '-e reports 1 error: missing files reported'
# at t/004_options.pl line 80.
# 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
# doesn't match '(?^:pg_xact.*is present in the manifest but not on disk)'

# Failed test 'nonexistent backup directory: matches'
# at t/004_options.pl line 86.
# 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
# doesn't match '(?^:could not open directory)'
# Looks like you failed 17 tests of 25.
t/004_options.pl ....... Dubious, test returned 17 (wstat 4352, 0x1100)
Failed 17/25 subtests
t/005_bad_manifest.pl .. 1/44
# Failed test 'missing pathname: matches'
# at t/005_bad_manifest.pl line 156.
# 'pg_validatebackup: fatal: could not parse backup
manifest: missing size
# '
# doesn't match '(?^:could not parse backup manifest: missing pathname)'

# Failed test 'missing size: matches'
# at t/005_bad_manifest.pl line 156.
# 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
# doesn't match '(?^:could not parse backup manifest: missing size)'

# Failed test 'file size is not an integer: matches'
# at t/005_bad_manifest.pl line 156.
# 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
# doesn't match '(?^:could not parse backup manifest: file size is
not an integer)'

# Failed test 'duplicate pathname in backup manifest: matches'
# at t/005_bad_manifest.pl line 156.
# 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
# doesn't match '(?^:fatal: duplicate pathname in backup manifest)'
t/005_bad_manifest.pl .. 31/44
# Failed test 'checksum without algorithm: matches'
# at t/005_bad_manifest.pl line 156.
# 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
# doesn't match '(?^:could not parse backup manifest: checksum
without algorithm)'

# Failed test 'unrecognized checksum algorithm: matches'
# at t/005_bad_manifest.pl line 156.
# 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
# doesn't match '(?^:fatal: unrecognized checksum algorithm)'

# Failed test 'invalid checksum for file: matches'
# at t/005_bad_manifest.pl line 156.
# 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
# doesn't match '(?^:fatal: invalid checksum for file)'
# Looks like you failed 7 tests of 44.
t/005_bad_manifest.pl .. Dubious, test returned 7 (wstat 1792, 0x700)
Failed 7/44 subtests

--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com