Re: Built-in Raft replication

On Wed, 16 Apr 2025 at 07:18, Ashutosh Bapat <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com>
wrote:

> On Wed, Apr 16, 2025 at 10:29 AM Andrey Borodin <x4mmm(at)yandex-team(dot)ru>
> wrote:
> >
> > If you use build-in failover you have to resort to 3 big Postgres
> machines because you need 2/3 majority. Of course, you can install
> MySQL-stype arbiter - host that had no real PGDATA, only participates in
> voting. But this is a solution to problem induced by built-in autofailover.
>
> Users find it a waste of resources to deploy 3 big PostgreSQL
> instances just for HA where 2 suffice even if they deploy 3
> lightweight DCS instances. Having only some of the nodes act as DCS
> and others purely PostgreSQL nodes will reduce waste of resources.
>
> The experience of other projects/products with automated failover based on
quorum shows that this is a critical issue for adoption. In the In-memory
Data Grid space (Coherence, Geode/GemFire) the question of how to ensure
that some nodes didn't carry any data comes up early in many architecture
discussions. When RabbitMQ shipped their Quorum Queues feature, the first
and hardest area of pushback was around all nodes hosting message content.

It's not just about the requirement for compute resources, it's also about
bandwidth and latency. Many large organisations have, for historical
reasons, pairs of data centres with very good point-to-point connectivity.
As the requirement for quorum witnesses has come up for all sorts of
things, including storage arrays, they have built arbiter/witness sites at
branches, colocation providers or even on the public cloud. More than not
holding user data or processing queries, the arbiter can't even be sent the
replication stream for the user data in the database, it just won't fit
down the pipe.

Which feels like a very difficult requirement to meet if the replication
model for all data is being changed to a quorum model.

Regards
Alastair