Re: Catalog/Metadata consistency during changeset extraction from wal

On Monday, June 25, 2012 08:50:54 PM Kevin Grittner wrote:
> Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> >> We most particularly *don't* want DDL to replicate automatically,
> >> because the schema changes are deployed along with related
> >> software changes, and we like to pilot any changes for at least a
> >> few days. Depending on the release, the rollout may take a
> >> couple months, or we may slam in out everywhere a few days after
> >> the first pilot deployment.
> >
> > Thats a sensible for your use-case - but I do not think its thats
> > the appropriate behaviour for anything which is somewhat
> > out-of-the box...

> Right. We currently consider the issues involved in a change and
> script it by hand. I think we want to continue to do that. The
> point was that, given the variety of timings and techniques for
> distributing schema changes, maybe is was only worth doing that
> automatically for the most constrained and controlled cases.
Agreed.

> >> So you could certainly punt all of this for any release as far as
> >> Wisconsin Courts are concerned. We need to know table and column
> >> names, before and after images, and some application-supplied
> >> metadata.
> >
> > I am not sure were going to get all that into 9.3.
>
> Sure, that was more related to why I was questioning how much these
> use cases even *could* integrate -- whether it even paid to
> *consider* these use cases at this point. Responses from Robert and
> you have pretty much convinced me that I was being overly
> pessimistic on that.
I think its an important question to ask, otherwise we might just end up with
infrastructure unusable for all our goals... Or usable but unfinished
infrastructure because its to complex to build in sensible time.

> One fine point regarding before and after images -- if a value
> doesn't change in an UPDATE, there's no reason to include it in both
> the BEFORE and AFTER tuple images, as long as we have the null
> column bitmaps -- or some other way of distinguishing unchanged from
> NULL. (This could be especially important when the unchanged column
> was a 50 MB bytea.) It doesn't matter to me which way this is
> optimized -- in our trigger-based system we chose to keep the full
> BEFORE tuple and just show AFTER values which were distinct from the
> corresponding BEFORE values, but it would be trivial to adapt the
> code to the other way.
I don't think doing that is worth the trouble in the first incarnation. There
is enough detail hidden in that that makes that non-trivial that I wouldn't
worry about it until the rest of the infrastructure exists. That part of the
code will definitely be version specific so I see no problem improving on that
incrementally.

> Sorry for that bout of pessimism.
Oh, no reason for that. I have some doubts about that myself, so...

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services