Re: Catalog/Metadata consistency during changeset extraction from wal

On 24 June 2012 22:50, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> On Sunday, June 24, 2012 11:37:26 PM Simon Riggs wrote:
>> On 24 June 2012 22:11, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
>> > One interesting problem are table rewrites (truncate, cluster, some ALTER
>> > TABLE's) and dropping tables. Because we nudge SnapshotNow to the past
>> > view it had back when the wal record was created we get the old
>> > relfilenode. Which might have been dropped in part of the transaction
>> > cleanup...
>> > With most types thats not a problem. Even things like records and arrays
>> > aren't problematic. More interesting cases include VACUUM FULL $systable
>> > (e.g. pg_enum) and vacuum full'ing a table which is used in the *_out
>> > function of a type (like a user level pg_enum implementation).
>
>> That's only a problem if you are generating changes to the relfilenode
>> rather than the relid.
> Hm. I can't follow so far. Could you paraphrase?

Basically, whether we generate SQL or not.

>> ISTM that this step differs depending upon whether we are generating
>> portable SQL, or whether we are generating changes for immediate
>> apply.
> I fear only generating changes for immediate, low-level apply is going to fly
> given the various interests people have voiced.

I'm OK with that, just checking what the objectives are.

>> If it is the latter, then it should never actually happen because if a table
>> rewrite occurred and then committed we would never need to re-read earlier
>> WAL.
>> So treating this as a generic problem leads to some weird results that
>> we don't need to worry about cos they can't actually happen.
> Well, even if it were true that we don't need to worry about the state before
> a full-table rewritte - I don't think it is - we still need to be able to cope
> with CLUSTER or VACUUM FULL...

If you have a WAL record for a new relfilenode, then you don't need to
read the catalog at all.

--
 Simon Riggs                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services