Re: Visibility regression test

On Thu, 2002-08-29 at 18:22, Tom Lane wrote:
> Rod Taylor <rbt(at)zort(dot)ca> writes:
> > Backend A locks row
> > Backends B and C are waiting on row
> > Backend A releases row
>
> > The problem is we cannot determine the order that B and C will wake up
> > in, which makes doing a diff against a standard case difficult.
>
> Exactly.
>

I've been wondering about the following (this is pseudocode, with the
main loop unrolled):

LOG("A.out","step 1");
LOG("B.out","step 1");
LOG("C.out","step 1");
PQsendQuery(A, ' whatever '); // Only executed if schedule lists one
PQsendQuery(B, ' whatever 2 ');
PQsendQuery(C, ' whatever 3 ');
sleep(n); // To allow queries to run. [See below for alternative]
PQconsumeInput(A);
if ! PQisbusy(A) {
LOG("A.out",PQgetResult(A)); // Actually a while(PQgetResult)
... etc. for B and C
LOG("A.out","step 2");
LOG("B.out","step 2");
LOG("C.out","step 2");
PQsendQuery(A, ' whatever next ');
... and so on.

This means that if A is holding a lock and B and C are waiting, their
step 1 output will be empty -the "step 1" marker will be logged,
followed by the "step 2" marker. If the next instruction in the schedule
file is a query which results in A releasing the lock, B and C's output
will be picked up and logged (in order of backend connection) between
the "step 2" and "step 3" log markers. Obviously, the schedule file
lists no queries for B and C in step 2 for this to work.

This means that the schedule file must have step markers for each state
that the system should be in -as that would be the only way to guarantee
that backend A had a particular command before backend (C). The sleep
statement above is a little crude (i.e. it might have to be set rather
long). In practice we could just loop until one backend returned, then
wait n seconds before testing all the backends again and moving to the
next stage. n could be much shorter in this case (representing the
likely difference in execution times between the backends) and would
probably be specified as a parameter of the test step.

If I get a chance I might try to write something on these lines.

> > We don't actually want to serialize the commands as that changes the
> > test.
>
> Good point. Maybe what we need is not so much emphasis on getting an
> exactly predetermined output, as a way of understanding the allowed
> variations in output order and making the tool able to complain just
> when unexpected variation occurs. In other words, something smarter
> than diff to check the results with.
>

This is another possibility, of course -and might allow for better
analysis of the results.

Regards

John
--
John Gray
Azuli IT
www.azuli.co.uk