Re: How can we expand PostgreSQL ecosystem?

On 6 March 2016 at 13:46, Mark Kirkwood <mark(dot)kirkwood(at)catalyst(dot)net(dot)nz>
wrote:

>
> For cloud - in particular Openstack (which I am working with ATM), the
> biggest thing would be:
>
> - multi-master replication
>

Working on it ;)

> or failing that:
>
> - self managing single master failover (voting/quorum etc)
>

I don't think that's either/or, frankly, in that you can have multimaster
that needs a lot of babying or multimaster that has magic autoscaling and
autosharding and autofailover (or some approximation of).

That said, I firmly believe that autofailover is a giant foot-cannon unless
*regularly tested in the live environment*. The same is true of most of the
other autoFOOs. Products that claim to offer them often do some creative
bending of the guarantees they claim to offer elsewhere, so I also tend to
take such claims with a grain or ten of salt. Witness the recent
discussions of Galleria.

Pg currently makes testing autofailover pretty painful, since failover is a
major and disruptive event that reduces your redundancy until the old
master has resynced. pg_rewind will help a lot there if it proves robust
enough - hopefully enough that we can start adopting a
ChaosMonkey-flavoured approach to HA.

Eventually, a long way down the track, I'd like to see logical replication
based failover be part of that picture too.

so that operators can essentially 'set and forget'. We currently use
> Mysql+ Galera (multi master) and Mongodb (self managing single master)
> and the convenience and simplicity is just so important (Openstack is a
> huge complex collection of services - hand holding of any one service is
> pretty much a non starter).
>

I rarely see a service where "self managing" doesn't turn into "needs lots
of tuning, tweaking and fixing" at significant scales, but it's nice to
have it be automagic and correct enough of the time for small to medium
deployments.

That's one area where Pg core's focus on getting it right can hold us back.
Not necessarily for the worse. We're not usually big on saying "oh, it
probably won't corrupt your data if you use it at reasonable transaction
rates with reasonable inter-node latencies". Having had a few interesting
surprises with
set-and-forget-until-it-brings-the-whole-network-down-and-you-have-no-idea-how-to-fix-it
systems I tend to favour doing it right myself. Which is why some of the
compromises currently necessary in the logical replication space on Pg irk
me so....

--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services