Re: Horizontal scalability/sharding

Hi,

On 09/03/2015 05:02 AM, Amit Kapila wrote:
> On Thu, Sep 3, 2015 at 8:28 AM, Bruce Momjian <bruce(at)momjian(dot)us
> <mailto:bruce(at)momjian(dot)us>> wrote:
> >
> > On Wed, Sep 2, 2015 at 07:50:25PM -0700, Joshua Drake wrote:
> > > >Can you explain why logical replication is better than binary
> > > >replication for this use-case?
> > > >
> > >
> > > Selectivity?
> >
>> I was assuming you would just create identical slaves to handle
>> failure, rather than moving selected data around.
> >
>
> Yes, I also think so, otherwise when the shard goes down and it's
> replica has to take the place of shard, it will take more time to
> make replica available as it won't have all the data as original
> shard had.

Not really, the idea is that you don't need to create the replica
immediately. The system recognizes that primary shard location is
unavailable and redirects the tasks to the "replicas." So the time to
recreate the failed node is not that critical.

It needs to be done in a smart way to prevent some typical issues like
suddenly doubling the load on replicas due to failure of the primary
location. By using different group of nodes for each "data segment" you
can eliminate this, because the group of nodes to handle the additional
load will be larger.

The other issue then of course is that the groups of nodes must not be
entirely random, otherwise the cluster would suffer data loss in case of
outage of arbitrary group of K nodes (where K is the number of replicas
for each piece of data).

It's also non-trivial to do this when you have to consider racks, data
centers etc.

With regular slaves you can't do any of this - no matter what you do,
you have to load balance the additional load only on the slaves.

regards

--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services