Re: Streaming replication and WAL archive interactions

On 04/22/2015 03:30 AM, Michael Paquier wrote:
> This is going to change a behavior that people are used to for a
> couple of releases. I would not mind having this patch do
> "archive_mode = on during recovery" => archive only segments generated
> by this node + the last partial segment on the old timeline at
> promotion, without renaming to preserve backward compatible behavior.
> If master and standby point to separate archive locations, this way
> the operator can sort out later the one he would want to use. If they
> point to the same location, archive_command scripts already do
> internally such renaming, at least that's what I suspect an
> experienced user would do, now it is true that not many people are
> experienced in this area I see mistakes regarding such things on a
> weekly basis... This .partial segment renaming is something that we
> should let the archive_command manage with its internal logic.

Currently, the archive command doesn't know if the segment it's
archiving is partial or not, so you can't put any logic there to manage
it. But if we archive it with the .partial suffix, then you can put
logic in the restore_command to check for .partial files, if you really
want to.

I feel that the best approach is to archive the last, partial segment,
but with the .partial suffix. I don't see any plausible real-world setup
where the current behaviour would be better. I don't really see much
need to archive the partial segment at all, but there's also no harm in
doing it, as long as it's clearly marked with the .partial suffix.

BTW, pg_receivexlog also uses a ".partial" file, while it's streaming
WAL from the server. The .partial suffix is removed when the segment is
complete. So there's some precedence to this. pg_receivexlog adds just
".partial" to the filename, it doesn't add any information of what
portion of the file is valid like I suggested here, though. Perhaps we
should follow pg_receivexlog's example at promotion too, for consistency.

- Heikki