Re: [HACKERS] Re: 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

On 2015-05-31 07:51:59 -0400, Robert Haas wrote:
> > 1) We continue determining the oldest SlruScanDirectory(SlruScanDirCbFindEarliest)
> > on the master to find the oldest offsets segment to
> > truncate. Alternatively, if we determine it to be safe, we could use
> > oldestMulti to find that.
> > 2) SlruScanDirCbRemoveMembers is changed to return the range of members
> > to remove, instead of doing itself
> > 3) We wal log [oldest offset segment guaranteed to not be alive,
> > nextmulti) for offsets, and [oldest members segment guaranteed to not be alive,
> > nextmultioff), and redo truncations for the entire range during
> > recovery.
> >
> > I'm pretty tired right now, but this sounds doable.
>
> I'm probably biased here, but I think we should finish reviewing,
> testing, and committing my patch before we embark on designing this.

Probably, yes. I am wondering whether doing this immediately won't end
up making some things simpler and more robust though.

> So far we have no reports of trouble attributable to the lack of the
> WAL-logging support discussed here, as opposed to several reports of
> trouble from the status quo within days of release.

The lack of WAL logging actually has caused problems in the 9.3.3 (?)
era, where we didn't do any truncation during recovery...

> By the time we've reached the minimum recovery point, they will have
> been recreated by the same WAL records that created them in the first
> place.

I'm not sure that's true. I think we could end up errorneously removing
files that were included in the base backup. Anyway, let's focus on your
patch for now.

> If, in the previous
> replay, we had wrapped all the way around, some of the stuff we keep
> may actually already have been overwritten by future WAL records, but
> they'll be overwritten again. Now, that could mess up our
> determination of which members to remove, I guess, but I'm not clear
> that actually matters either: if the offsets space has wrapped around,
> the members space will certainly have wrapped around as well, so we
> can remove anything we like at this stage and we're still OK. I agree
> this is ugly the way it is, but where is the actual bug?

I'm more worried about the cases where we didn't ever actually "badly
wrap around" (i.e. overwrite needed data); but where that's not clear on
the standby because the base backup isn't in a consistent state.

Greetings,

Andres Freund