Re: standard interfaces for replication providers

Markus Schiltknecht wrote:
> Hi,
>
> José Orlando Pereira wrote:
>> I would argue that people haven't been able to build production-grade
>> multi-master replication, in part, due to the barrier of not having a
>> "standard" database-agnostic API. :-)
>>
>> In fact, the problem is not the lack of a "standard" API but the lack
>> of an API at all. Having to learn the intrincacies of a database
>> server (or build a full fledged server wrapper) to experiment with
>> simple prototypes is a serious hurdle for R&D in database
>> replication. This has been the case for replication based on group
>> communication.
>
> I disagree. There have been a couple of approaches and each has had a
> different interface to the database. But for most of them, coding the
> database interface was _not_ the hardest part. And a good
> understanding of the database system internals is simply required to
> write a good replication system.
I agree with you.
But, I would add that the same understanding is required to design and
implement this interface on any database system.
>
>> Please don't confuse two proposals included in the distributed package:
>>
>> (1) A PostgreSQL specific patch, which implements a minimal set of
>> required features with a PostgreSQL-specific interface (e.g.
>> triggers, new statements, configuration variables). This is by no
>> means a "standard" interface. The included technical report discusses
>> why these are required for a variety of replication scenarios.
>
> Where do I find the included technical report?
GordaInterfaces/javasrc/docs/gapi.pdf
>
> I've read the READMEs in PostgreSQL/G toplevel and csrc directory and
> did not find convincing reasons why exactly these triggers need to be
> added as replication hooks. In fact, Postgres-R (8) would already need
> different hooks.
Such triggers are not necessary to develop any kind of replication
protocol... However, indeed any replication protocol requires a set of
hooks that might be enabled by different means, e.g. call back
functions, triggers, etc, etc...
We developed our hooks by means of triggers as their provide a standard
interface (triggers) that might be easily exploited by other projects
besides replication, e.g. materialized views.

>
> From studying the patch, I understand that these hooks are quite close
> to what's needed for a Postgres-R or Slony-II like sync, multi-master
> replication system (i.e. hooks for writeset extraction, the addition
> of a 'serialization error' for remote transactions).
>
> I can see use for such an API as soon as we have a production-grade
> replication system, which performs well enough in most applications
> (i.e. when we know exactly where to place the hooks). But up until
> then, people will try different algorithms and different hooks.
>
> Concerning my work on Postgres-R I can tell: I'm not going to use
> these triggers (hooks) because they are limiting.
Could you tell me why they are limiting ?
> I know enough about the database system internals and I _want_ to
> fiddle with the database system. Why should I use such an API?
The idea is not to impose our API but what we really want is to show
that behind the requirements for replication systems there is place for
a variety of systems that could be leverage by means of a "standard api".

Jose Orlando, any comments on that ?