From: | Jesper Pedersen <jesper(dot)pedersen(at)comcast(dot)net> |
---|---|
To: | Jelte Fennema-Nio <postgres(at)jeltef(dot)nl>, Ants Aasma <ants(dot)aasma(at)cybertec(dot)at> |
Cc: | Peter Eisentraut <peter(at)eisentraut(dot)org>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: protocol-level wait-for-LSN |
Date: | 2024-10-30 18:03:48 |
Message-ID: | 78cc73fe-261b-4854-8ae9-5edfb9e47f00@comcast.net |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
Hi,
On 10/30/24 1:45 PM, Jelte Fennema-Nio wrote:
> On Wed, 30 Oct 2024 at 18:18, Ants Aasma <ants(dot)aasma(at)cybertec(dot)at> wrote:
>> The idea is great, I have been wanting something like this for a long
>> time. For future proofing it might be a good idea to not require the
>> communicated-waited value to be a LSN.
>
> Yours and Matthias' feedback make total sense I think. From an
> implementation perspective I think there are a few things necessary to
> enable these wider usecases:
> 1. The token should be considered opaque for clients (should be documented)
> 2. The token should be defined as variable length in the protocol
> 3. We should have a hook to allow postgres extensions to override the
> default token generation
> 4. We should have a hook to allow postgres extensions to override
> waiting until the token "timestamp"
>
>> Even without sharding LSN might not be a final choice. Right now on
>> the primary the visibility order is not LSN order. So if a connection
>> does synchronous_commit = off commit, the write location is not even
>> going to see the commit. By publishing the end of the commit record it
>> would be better. But I assume at some point we would like to have a
>> consistent visibility order, which quite likely means using something
>> other than LSN as the logical clock.
>
> I was going to say that the default could probably still be LSN, but
> this makes me doubt that. Is there some other token that we can send
> now that we could "wait" on instead of the LSN, which would work for.
> If not, I think LSN is still probably a good choice as the default. Or
> maybe only as a default in case synchronous_commit != off.
>
There are known wish-lists for a protocol v4, like
https://github.com/pgjdbc/pgjdbc/blob/master/backend_protocol_v4_wanted_features.md
and a lot of clean-room implementations in drivers and embedded in
projects/products.
Having LSN would be nice, but to break all existing implementations, no.
Having to specify with startup parameters how a core message format
looks like sounds like a bad idea to me,
https://www.postgresql.org/docs/devel/protocol-message-formats.html
is it.
If we want to start on a protocol v4 thing then that is ok - but there
are a lot of feature requests for that one.
Best regards,
Jesper
From | Date | Subject | |
---|---|---|---|
Next Message | Andres Freund | 2024-10-30 18:11:52 | Re: AIO writes vs hint bits vs checksums |
Previous Message | Bharath Rupireddy | 2024-10-30 17:51:23 | Re: Introduce new multi insert Table AM and improve performance of various SQL commands with it for Heap AM |