Re: [PATCH 14/16] Add module to apply changes from an apply-cache using low-level functions

On Wed, Jun 13, 2012 at 7:28 AM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> From: Andres Freund <andres(at)anarazel(dot)de>
>
> We decided to use low level functions to do the apply instead of producing sql
> statements containing the data (or using prepared statements) because both, the
> text conversion and the full executor overhead aren't introduce a significant
> overhead which is unneccesary if youre using the same version of pg on the same
> architecture.
>
> There are loads of use cases though that require different methods of applyin
> though - so the part doing the applying from an ApplyCache is just a bunch of
> well abstracted callbacks getting passed all the required knowledge to change
> the data representation into other formats.

It's important to make sure that it's not going to be *too* difficult
to "jump through the hoops" necessary to apply changes on a different
version.

While pg_upgrade has diminished the need to use replication to handle
cross-version/architecture upgrades, I don't think it has brought that
to zero.

One other complication I'll observe... The assumption is being made
that UPDATE/DELETE will be handled via The Primary Key. For the most
part, I approve of this. Once upon a time, Slony had a misfeature
where you could tell it to add in a surrogate primary key, and that
caused no end of trouble. However, the alternative, that *does* seem
to work alright, is to allow selecting a candidate primary key, that
is, a set of columns that have UNIQUE + NOT NULL characteristics. I
could see people having a problem switching over to use this system if
they MUST begin with a 'Right Proper Primary Key.' If they start with
a system with a 2TB table full of data that lacks that particular
constraint, that could render them unable to use the facility.

> Missing:
>
> - TOAST handling. For physical apply not much needs to be done because the
>  toast inserts will have been made beforehand. There needs to be an option in
>  ApplyCache that helps reassembling TOAST datums to make it easier to write
>  apply modules which convert to text.

Dumb question: Is it possible that two nodes would have a different
idea as to which columns need to get toasted? I should think it
possible for nodes to be configured with different values for TOAST
policies, and while it's likely pretty dumb to set them to have
different handlings, it would seem undesirable to not bother looking,
and find the backend crashing due to an un-noticed mismatch.

--
When confronted by a difficult problem, solve it by reducing it to the
question, "How would the Lone Ranger handle this?"