Re: Core team statement on replication in PostgreSQL

On 5/29/08, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> The Postgres core team met at PGCon to discuss a few issues, the largest
> of which is the need for simple, built-in replication for PostgreSQL.
> Historically the project policy has been to avoid putting replication
> into core PostgreSQL, so as to leave room for development of competing
> solutions, recognizing that there is no "one size fits all" replication
> solution. However, it is becoming clear that this policy is hindering
> acceptance of PostgreSQL to too great an extent, compared to the benefit
> it offers to the add-on replication projects. Users who might consider
> PostgreSQL are choosing other database systems because our existing
> replication options are too complex to install and use for simple cases.
> In practice, simple asynchronous single-master-multiple-slave
> replication covers a respectable fraction of use cases, so we have
> concluded that we should allow such a feature to be included in the core
> project. We emphasize that this is not meant to prevent continued
> development of add-on replication projects that cover more complex use
> cases.
>
> We believe that the most appropriate base technology for this is
> probably real-time WAL log shipping, as was demoed by NTT OSS at PGCon.
> We hope that such a feature can be completed for 8.4.

+1

Although I would explain it more shortly - we do need a solution for
lossless failover servers and such solution needs to live in core backend.

> Ideally this
> would be coupled with the ability to execute read-only queries on the
> slave servers, but we see technical difficulties that might prevent that
> from being completed before 8.5 or even further out. (The big problem
> is that long-running slave-side queries might still need tuples that are
> vacuumable on the master, and so replication of vacuuming actions would
> cause the slave's queries to deliver wrong answers.)

Well, both Slony-I and upcoming Skytools 3 have the same problem when
cleaning events and have it solved simply by slaves reporting back their
lowest position on event stream. I cannot see why it cannot be applied
in this case too. So each slave just needs to report its own longest
open tx as "open" to master. Yes, it bloats master but no way around it.

Only problem could be the plan to vacuum tuples updated in between long
running tx and the regular ones, but such behaviour can be just turned off.

We could also have a option of "inaccessible slave", for those who
fear bloat on master.

--
marko