Re: Reviewing freeze map code

On 2016-05-06 18:36:52 -0300, Alvaro Herrera wrote:
> Andres Freund wrote:
>
> > On 2016-05-06 14:17:13 -0700, Joshua D. Drake wrote:
> > > How do I test?
> > >
> > > Is there a script I can run?
> >
> > Unfortunately there's few interesting things to test with pre-made
> > scripts. There's no relevant OS dependency here, so each already
> > existing test doesn't really lead to significantly increased coverage
> > being run by other people. Generally, when testing for correctness
> > issues, it's often of limited benefit to run tests written by the author
> > of reviewer - such scripts will usually just test things either has
> > thought of. The dangerous areas are the ones neither author or reviewer
> > has considered.
>
> We touched this question in connection with multixact freezing and
> wraparound. Testers seem to want to be given a script that they can
> install and run, then go for a beer and get back to a bunch of errors to
> report. But it doesn't work that way; writing a useful test script
> requires a lot of effort.

Right. And once written, often enough running it on a lot more instances
only marginally increases the coverage.

> Jeff Janes has done astounding work in these matters. (I don't think
> we credit him enough for that.)

+many.