Re: Database cluster?

> > The proxy monitors and forwards the requests from the clients to the
> > database servers. If it is a read-only request the query is forwarded
> > to the databaseserver currently experiencing the lowest load/most free
> > memory, otherwise it is sent to all database servers.
> >
> > This approach obviously only performs well in systems with a high ratio
> > of read-only queries, such as search engines and so on.
>
> The tough part is syncronicity, should one of the machines drop out of
> the cluster and need to be re-added without bringing the others down.
> In order to get around that, each query needs to be logged on the master
> node with a timestamp, so that the failed node can "catch up" in
> real-time. That brings about other considerations, as well....

What about using something like RAID-type setup on the database servers? So
that for example, all data (records or tables, depending on the split)
exists on more than one server. That way if a server goes down, all the
data would still exist on the other servers. A server failure could trigger
a replication operation that would re-distribute the data across the
remaining servers in such a way that if another one fails, the data
integrity would still be preserved. Similarly, when a new server is added,
this should trigger an operation that would re-distribute records in sich a
way as to insure redundancy with the new number of server (in this case,
move some records from the old servers to the new one and load balance it).
This, again, "shouldn't" be too hard to implement, if we could sort out the
design first.

However, this could be (theoretically) difficult to combine with
"near-optimal" table/record distribution. The two might go against each
other. There should be a way of enforcing the redundancy part, and then
doing a "best effort" to do it in such a way as to gain most performance
improvement. This _could_ be done, and off the top of my head, I cannot see
any reason why it wouldn't work reasonably well. Obviously, a server
failure would cause the whole thing to slow down for a while until the
records were re-distributed, and the optimizer re-assesses the situation,
but that is true of any redundant system that has to remain at least
operational during a failure. After the replication is completed, it should
resume at it's full speed (dictated by the number of available servers).

Any opinions on this?

Regards.

Gordan