Re: Batch replication ordering (was Re: [GENERAL] 32/64-bit

On Fri, Apr 11, 2003 at 02:18:41PM -0600, Ed L. wrote:
> Suppose further you wish to limit the number of updates replicated in a
> particular cycle. For example, suppose you have a million changes queued
> for replication on the master, but for obvious reasons you don't want to
> select a million rows to replicate all at once. Suppose you also don't
> want to grab them one tuple or one transaction at a time, preferring to
> avoid hammering the master. Rather, you want to grab them in batches of no
> more than N transactions, replicate them all to a slave and commit on the
> slave, take a breather, repeating until the slave is caught up. And during
> each breather, you want to have committed only complete transactions so
> that any slave clients see consistent data.

I know I'm a bit ignorant to how dbmirror works, but why do the select
from the master and the actual commits on the slave(s) have to occupy a
single transaction? I would expect that there was some kind of
book-keeping tables on both the master and the slaves to keep track of
how far along mirroring/replication was; the master would need to know
what it could remove out of the replication table (table Q as it's
called elsewhere in this thread), and the slave needs to know where it
left off. If this is the case, isn't it acceptable to select a chunk of
data out of the queue table on the master, and run through it on the
slave, with the slave committing as it pleases? Of course every time the
slave commits it will have to update the book-keeping tables, but that
seems to be a given... or is this exactly what you're worried about
hammering the master with?
--
Jim C. Nasby (aka Decibel!) jim(at)nasby(dot)net
Member: Triangle Fraternity, Sports Car Club of America
Give your computer some brain candy! www.distributed.net Team #1828

Windows: "Where do you want to go today?"
Linux: "Where do you want to go tomorrow?"
FreeBSD: "Are you guys coming, or what?"