Re: Should we remove "not fast" promotion at all?

On 08/08/2013 10:34 AM, Andres Freund wrote:
> On 2013-08-08 10:15:14 -0700, Josh Berkus wrote:
>> Either we have confidence is fast promotion, or we don't. If we don't
>> have confidence, then either (a) more testing is needed, or (b) it
>> shouldn't be the default. Again, here, we are coming up against our
>> lack of any kind of broad replication failure testing.
>
> While I think we definitely miss out there I don't think any regression
> suite would help much here. I am wary of unknown problems, not ones
> we already have tests for. The subtle ones aren't easy to test, even
> with a regression suite.

Yeah, that's why we have to get beyond the mentality that regression
testing is the only kind of testing. We need a destruction test for
replication, and that's NOT going to be a regression test. Among other
things, we'll probably need to run it on cloud hosting.

> The problem is that, especially involving HS, there's lots of subtle
> corner cases. And those are pretty hard to forsee and thus hard to
> test.

It would be useful to assemble a list of corner cases we *do* know
about. This could become a test suite, and we could keep adding to it.

> Being able to tell somebody to touch some file and kill a certain
> process instead of pg_ctl triggering is certainly better than to have
> them apply complex patches which then only exhibit the old behaviour.
> It's not about letting people regularly use it or such. It's about being
> able to verify problems.

The problem is, if failover fails badly, the user is probably facing a
corrupt database, downtime, loss of data, and restore from backup. So
if we don't think that fast failover is rock-solid trustworthy --- or at
least as trustworthy as slow failover was -- then we should be making it
a non-default option for 9.3. We shouldn't be exposing people who don't
need fast failover to new risks without their knowledge.

--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com