Re: multithreaded zstd backup compression for client and server

From: Andrew Dunstan <andrew(at)dunslane(dot)net>
To: Dagfinn Ilmari Mannsåker <ilmari(at)ilmari(dot)org>, Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: Dipesh Pandit <dipesh(dot)pandit(at)gmail(dot)com>, Andres Freund <andres(at)anarazel(dot)de>, Justin Pryzby <pryzby(at)telsasoft(dot)com>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: multithreaded zstd backup compression for client and server
Date: 2022-03-30 12:55:30
Message-ID: f74072e4-f1df-2669-a856-abf38bc35811@dunslane.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers


On 3/30/22 08:06, Dagfinn Ilmari Mannsåker wrote:
> Robert Haas <robertmhaas(at)gmail(dot)com> writes:
>
>> On Mon, Mar 28, 2022 at 12:52 PM Dagfinn Ilmari Mannsåker
>> <ilmari(at)ilmari(dot)org> wrote:
>>>> True, but that also means it shows up in the actual failure message,
>>>> which seems too verbose. By just using 'print', it ends up in the log
>>>> file if it's needed, but not anywhere else. Maybe there's a better way
>>>> to do this, but I don't think using note() is what I want.
>>> That is the difference between note() and diag(): note() prints to
>>> stdout so is not visible under a non-verbose prove run, while diag()
>>> prints to stderr so it's always visible.
>> OK, but print doesn't do either of those things. The output only shows
>> up in the log file, even with --verbose. Here's an example of what the
>> log file looks like:
>>
>> # Running: pg_verifybackup -n -m
>> /Users/rhaas/pgsql/src/bin/pg_verifybackup/tmp_check/t_008_untar_primary_data/backup/server-backup/backup_manifest
>> -e /Users/rhaas/pgsql/src/bin/pg_verifybackup/tmp_check/t_008_untar_primary_data/backup/extracted-backup
>> backup successfully verified
>> ok 6 - verify backup, compression gzip
>>
>> As you can see, there is a line here that does not begin with #. That
>> line is the standard output of a command that was run by the test
>> script.
> Oh, that must be some non-standard output handling that our test setup
> does. Plain `prove` shows everything on stdout and stderr in verbose
> mode, and only stderr in non-vebose mode:
>

Yes, PostgreSQL::Test::Utils hijacks STDOUT and STDERR (see the INIT
block).

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Magnus Hagander 2022-03-30 13:04:30 Re: Add parameter jit_warn_above_fraction
Previous Message Robert Haas 2022-03-30 12:53:43 Re: pgsql: Add 'basebackup_to_shell' contrib module.