From: | Jeff Janes <jeff(dot)janes(at)gmail(dot)com> |
---|---|
To: | pgsql-hackers <pgsql-hackers(at)postgresql(dot)org> |
Subject: | pg_basebackup delays closing of stdout |
Date: | 2019-07-24 02:16:26 |
Message-ID: | CAMkU=1xiF0kxmCs_w7b1pp9G0pbS5-0FtOJrros3NpU21PWmaw@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
Ever since pg_basebackup was created, it had a comment like this:
* End of chunk. If requested, and this is the base tablespace
* write configuration file into the tarfile. When done, close the
* file (but not stdout).
But, why make the exception for output going to stdout? If we are done
with it, why not close it?
After a massive maintenance operation, I want to re-seed a streaming
standby, which I start to do by:
pg_basebackup -D - -Ft -P -X none | pxz > base.tar.xz
But the archiver is way behind, so when it finishes the basebackup part, I
get:
NOTICE: pg_stop_backup cleanup done, waiting for required WAL segments to
be archived
WARNING: pg_stop_backup still waiting for all required WAL segments to be
archived (60 seconds elapsed)
...
The base backup file is not finalized, because pg_basebackup has not closed
its stdout while waiting for the WAL segment to be archived. The file is
incomplete due to data stuck in buffers, so I can't copy it to where I want
and bring up a new streaming replica (which bypasses the WAL archive, so
would otherwise work). Also, if pg_basebackup gets interupted somehow while
it is waiting for WAL archiving, the backup will be invalid, as it won't
flush the last bit of data. Of course if it gets interupted, I would have
to test the backup to make sure it is valid. But testing it and finding
that it is valid is better than testing it and finding that it is not.
I think it makes sense for pg_basebackup to wait for the WAL to be
archived, but there is no reason for it to hold the base.tar.xz file
hostage while it does so.
If I simply remove the test for strcmp(basedir, "-"), as in the attached, I
get the behavior I desire, and nothing bad seems to happen. Meaning, "make
check -C src/bin/pg_basebackup/" still passes (but only tested on Linux).
Is there a reason not to do this?
Cheers,
Jeff
Attachment | Content-Type | Size |
---|---|---|
pg_basebackup_close_stdout.patch | application/octet-stream | 1022 bytes |
From | Date | Subject | |
---|---|---|---|
Next Message | Michael Paquier | 2019-07-24 02:29:28 | Re: pg_receivewal documentation |
Previous Message | David Rader | 2019-07-24 01:59:52 | Re: Procedure support improvements |