Re: Issue with markers in isolation tester? Or not?

On Sun, Jan 19, 2025 at 07:26:58PM +0100, Michail Nikolaev wrote:
> Mostly idea is next:
>
> Let's imagine we have two steps - step_before and step_after which may end
> in either order.
> Then instead of such step/markers structure:
>
> step_before(step_after)
> stepN
> stepN+1
> step_after
>
> use the next:
>
> step_before
> stepN
> stepN+1
> step_after(step_before)
>
> In the first case, there are two possible results:
> 1) step_before is finished before step_after - reported as step_before,
> step_after
> 2) step_after is launched before step_before ends - reported as step_after,
> step_before
>
> But in the case second variant:
> 1) step_before is finished before step_after - reported as step_before,
> step_after
> 2) step_after is launched before step_before ends - reported as
> step_before, step_after
>
> So, the second option provides the same result regardless of order of
> finishing of step_before and step_after, which is the thing I want to
> achieve here.

Stepping back, any variation in the total ordering of step-start and
step-complete events necessarily changes the expected output. Hence, we want
to freeze that ordering. We could do that directly, by having isolationtester
read a format that lists start and complete events. We could also have the
isolationtester write the same format, so you could draft a permutation
without blocking conditions and just save the order that isolationtester
discovers by experiment. I'd find that less mentally taxing than following
rules to reach a frozen ordering via blocking conditions.

As an example of one concrete way to achieve that, have isolationtester write
a results/SPECNAME.wait file that contains syntax like "permutation-event
waiting:S1 S2 complete:S1 S3". You'd copy that file to
expected/SPECNAME.wait, which isolationtester would read at startup. This
wouldn't replace "notices" blocking conditions for certain complex tests, and
tests with multiple expected outputs might not want it. It would cover the
rest. Any unprefixed entry (e.g. the plain "S2") would get the simplification
of skipping the check for whether the step has entered a wait.

How well would that meet your test's needs?