Re: Applying logical replication changes by more than one process

On Mar 22, 2016, at 11:14 AM, Petr Jelinek wrote:
>
> And each slot means connection with logical decoding attached to it so you don't really want to have thousands of those anyway. I think you'll hit other problems faster than loop over slots becomes problem if you plan to keep all of them active.

Assume that cluster have thousands of nodes and we use sharding to scatter data through cluster nodes.
But to provide HA we want to perform sharding with some level of redundancy, for example save the same record at 3 different nodes.
Once possible approach (pg_shard) is to execute the same query at three different shards.
But there is no warranty that result of execution will be the same at all nodes.
Alternative approach is to execute transaction at one node and then replicate it using logical replication to replicas.
So we do not perform logical replication to all 1000 nodes. Just to 2 of them. But each time it will be different pair of nodes. So we still need to have 1000 active replication slots.

May be logical replication can not be used at all in such scenario - I have not thought much about it yet. Our first step will be multimaster without sharding.

>
> --
> Petr Jelinek http://www.2ndQuadrant.com/
> PostgreSQL Development, 24x7 Support, Training & Services
>
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers