Re: repeated decoding of prepared transactions

On 22.02.21 05:22, Andres Freund wrote:
> Hi,
>
> On 2021-02-19 15:53:32 +0100, Markus Wanner wrote:
>> However, more generally speaking, I suspect you are overthinking this. All
>> of the complexity arises because of the assumption that an output plugin
>> receiving and confirming a PREPARE may not be able to persist that first
>> phase of transaction application. Instead, you are trying to somehow
>> resurrect the transactional changes and the prepare at COMMIT PREPARED time
>> and decode it in a deferred way.
>
> The output plugin should never persist anything.

Sure, sorry, I was sloppy in formulation. I meant the replica or client
that receives the data from the output plugin. Given it asked for
two-phase commits in the output plugin, it clearly is interested in the
PREPARE.

> That's the job of the
> client, not the output plugin. The output plugin simply doesn't have the
> information to know whether the client received data and successfully
> applied data or not.

Exactly. Therefore, it should not randomly reshuffle or reorder
PREPAREs until after other COMMITs.

> The output plugin doesn't set / influence start_decoding_at (unless you
> want to count just ERRORing out).

Yeah, same sloppiness, sorry.

>> With that line of thinking, the point in time (or in WAL) of the COMMIT
>> PREPARED does not matter at all to reason about the decoding of the PREPARE
>> operation. Instead, there are only exactly two cases to consider:
>>
>> a) the PREPARE happened before the start_decoding_at LSN and must not be
>> decoded. (But the effects of the PREPARE must then be included in the
>> initial synchronization. If that's not supported, the output plugin should
>> not enable two-phase commit.)
>
> I don't think that can be made work without disproportionate
> complexity. Especially not in cases where we start to be CONSISTENT
> based on pre-existing on-disk snapshots.

Well, the PREPARE to happen before the start_decoding_at LSN is a case
the output plugin needs to deal with. I pointed out why the current way
of dealing with it clearly is wrong.

What issues do you see with the approach I proposed?

Regards

Markus