Re: SQL workflow for crash testing correctness

Thanks Jeff!

On Wed, Sep 18, 2019 at 2:39 PM Jeff Janes <jeff(dot)janes(at)gmail(dot)com> wrote:

> On Tue, Sep 17, 2019 at 9:27 PM Joseph Hammerman <
> jhammerman(at)squarespace(dot)com> wrote:
>
>> Good evening PGSQL admin email distribution list,
>>
>> I have built an HA cluster setup. I would like to instrument a workflow
>> to test for lost or duplicated writes.
>>
>> Does anyone know of prior art that does this?
>>
>>
> I have a testing framework which injects faults under high load, and then
> tests to see that automatic recovery happens correctly. I have used it to
> find several bugs, but haven't turned up any in the last couple releases
> (likely because improved regression tests are now catching them before I
> get a chance to). I've always just tested this as crash recovery within a
> single instance, but I think there is no reason the technique couldn't be
> used for multiple instances is well. You can search for my name and "
> count.pl" on the hackers list to find multiple example of the testing
> harness. The nature of the fault injected (torn page writes) is just a
> function of what I was working on at the time I wrote it, most of the bugs
> uncovered had nothing to do with the exact thing which caused the crash.
>
>
>
>> Does anyone have thoughts on how to model this? My initial thoughts were
>> to find the serialization tests in the Postgres project core.
>>
>
> Looking at the core regression tests may also be a good idea. Of course
> then you would have to ponder, if you test the same way as they do, will
> you find different bugs from what they find? So I would view it more as
> inspiration than as instructions.
>
> Cheers,
>
> Jeff
>
>>