Re: Synchronous replication

On Tue, Jul 27, 2010 at 01:41:10PM +0900, Fujii Masao wrote:
> On Tue, Jul 27, 2010 at 12:36 PM, Joshua Tolley <eggyknap(at)gmail(dot)com> wrote:
> > Perhaps I'm hijacking the wrong thread for this, but I wonder if the quorum
> > idea is really the best thing for us. I've been thinking about Oracle's way of
> > doing things[1]. In short, there are three different modes: availability,
> > performance, and protection. "Protection" appears to mean that at least one
> > standby has applied the log; "availability" means at least one standby has
> > received the log info (it doesn't specify whether that info has been fsynced
> > or applied, but presumably does not mean "applied", since it's distinct from
> > "protection" mode); "performance" means replication is asynchronous. I'm not
> > sure this method is perfect, but it might be simpler than the quorum behavior
> > that has been considered, and adequate for actual use cases.
>
> In my case, I'd like to set up one synchronous standby on the near rack for
> high-availability, and one asynchronous standby on the remote site for disaster
> recovery. Can Oracle's way cover the case?

I don't think it can support the case you're interested in, though I'm not
terribly expert on it. I'm definitely not arguing for the syntax Oracle uses,
or something similar; I much prefer the flexibility we're proposing, and agree
with Yeb Havinga in another email who suggests we spell out in documentation
some recipes for achieving various possible scenarios given whatever GUCs we
settle on.

> "availability" mode with two standbys might create a sort of similar situation.
> That is, since the ACK from the near standby arrives in first, the near standby
> acts synchronous and the remote one does asynchronous. But the ACK from the
> remote standby can arrive in first, so it's not guaranteed that the near standby
> has received the log info before transaction commit returns a "success" to the
> client. In this case, we have to failover to the remote standby even if it's not
> under control of a clusterware. This is a problem for me.

My concern is that in a quorum system, if the quorum number is less than the
total number of replicas, there's no way to know *which* replicas composed the
quorum for any given transaction, so we can't know which servers to fail to if
the master dies. This isn't different from Oracle, where it looks like
essentially the "quorum" value is always 1. Your scenario shows that all
replicas are not created equal, and that sometimes we'll be interested in WAL
getting committed on a specific subset of the available servers. If I had two
nearby replicas called X and Y, and one at a remote site called Z, for
instance, I'd set quorum to 2, but really I'd want to say "wait for server X
and Y before committing, but don't worry about Z".

I have no idea how to set up our GUCs to encode a situation like that :)

--
Joshua Tolley / eggyknap
End Point Corporation
http://www.endpoint.com