Re: protocol-level wait-for-LSN

On 30/10/2024 13:34, Tatsuo Ishii wrote:
>> On Wed, 30 Oct 2024 at 07:49, Tatsuo Ishii <ishii(at)postgresql(dot)org> wrote:
>>>> I think one would have to define that somehow. If it's useful, the
>>>> additional fields of both extensions could be appended, in some
>>>> defined order. But this is an interesting question to think about.
>>>
>>> I think this kind of extension, which adds new filed to an existing
>>> message type, should be implemented as v4 protocol.
>>
>> Could you explain why you think a major version bump is needed? In
>> what situation do you care about this. Because for my usecases (client
>> implementations & pgbouncer) I don't think that would be necessary. If
>> a client doesn't send the _pq_.wait_for_lsn protocol parameter, it
>> will never receive this new version.
>
> Yes, if there's only one extension for a message type, it would not be
> a big problem. But if there's more than one extensions that want to
> change the same type, problem arises as I have already discussed them
> upthread.
>
>> I don't really see a problem with having two protocol parameters
>> change the same message. Yes, you have to define what the result of
>> their combination is, but that seems trivial to do for additions of
>> fields. You either define the first protocol parameter that was added
>> to the spec, to add its field before the second. Or you could do it
>> based on something non-time-dependent, like the alphabetic order of
>> the protocol parameter, or the alphabetic order of the fields that
>> they add.
>
> That sounds far from trivial. So each extension needs to check if any
> other extension which modifies the same message type is activated?
> That requires each extension implementation to have built-in knowledge
> about any conflicting extension. Moreover each extension may not be
> added at once. If extension Y is added after extension X is defined,
> then implementation of X needs to be changed because at the time when
> X is defined, it did not need to care about Y. Another way to deal
> with the problem could be defining a new protocol message which
> describes those conflict information so that each extensions do not
> need to have such information built-in, but maybe it is too complex.

Note that the "protocol extension" mechanism is *not* meant for
user-defined extensions. That's not the primary purpose anyway. It
allows evolving the protocol in core code in a backwards compatible way,
but indeed the different extensions will need to be coordinated so that
they don't clash with each other. If they introduced new message types
for example, they better use different message type codes.

We might have made a mistake by calling this mechanism "protocol
extensions", because it makes people think of user-defined extensions.

With user-defined extensions, yes, you have exactly the problem you
describe.We have no rules on how a protocol extension is allowed to
change the protocol. It might add fields, it might add messages, or it
might change the meaning of existing messages. Or encapsulate the whole
protocol in XML.

So yes, each protocol extension needs to know about all the other
protocol extensions that it can be used with. In practice we'll avoid
doing crazy stuff so that the protocol extensions are orthogonal, but if
user-defined extensions get involved, there's not much we can do to
ensure that.

--
Heikki Linnakangas
Neon (https://neon.tech)