Archiving last incomplete segment as .partial issues

From: Heikki Linnakangas <hlinnaka(at)iki(dot)fi>
To: pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Archiving last incomplete segment as .partial issues
Date: 2015-05-21 12:35:13
Message-ID: 555DD101.7080209@iki.fi
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

I noticed that my patch to archive the last incomplete segment from old
timeline at promotion with the .partial suffix (de768844) was a few
bricks shy of a load. It makes a copy of the segment with the .partial
suffix, and it gets archived correctly, but it still leaves the segment
lying in pg_xlog. After enough time has passed that the segment becomes
old enough to be recycled, it will still be archived, without the
.partial suffix, which has all the same problems as before.

To fix, the old segment should be renamed rather than copied, to have
the .partial suffix. And that needs to be done later in the startup
sequence, after the end-of-recovery record has been written, because if
the server crashes before that, it still needs the partial segment to
recover.

Attached is a patch to do that.

Another option would be to create a .done file for the last partial
segment immediately after the .partial copy has been made, so that it
won't get archived, but I think it's weird to have a .done file for a
segment that hasn't in fact been archived.

In the original commit, I refactored XLogFileCopy() to not call
InstallXLogFileSegment(), leaving that to the caller. But with the
attached patch, that refactoring is no longer needed, and could be
reverted. I think it still makes sense, from a code readability point of
view, although I wouldn't have bothered if it wasn't needed by the
original patch. Thoughts? I'm inclined to not revert the XLogFileCopy()
changes, although reverting might make backporting future patches
slightly easier.

- Heikki

Attachment Content-Type Size
0001-At-promotion-don-t-leave-behind-a-partial-segment-on.patch application/x-patch 8.6 KB

Browse pgsql-hackers by date

  From Date Subject
Next Message Robert Haas 2015-05-21 12:38:33 Re: Parallel Seq Scan
Previous Message Dean Rasheed 2015-05-21 12:22:18 Re: GROUPING