Re: Clock-skew management in logical replication

On 9/21/24 01:31, shihao zhong wrote:
> Nisha Moond <nisha(dot)moond412(at)gmail(dot)com> writes:
>> Thoughts? Looking forward to hearing others' opinions!
>
> Had a productive conversation with Amit Kaplia today about time skew
> in distributed systems, and wanted to share some thoughts.
> Essentially, we're grappling with the classic distributed snapshot
> problem. In a multi-active environment, where multiple nodes can
> independently process transactions, it becomes crucial to determine
> the visibility of these transactions across the system. Time skew,
> where different machines have different timestamps make it a hard
> problem. How can we ensure consistent transaction ordering and
> visibility when time itself is unreliable?
>
> As you mentioned, there are several ways to tackle the time skew
> problem in distributed systems. These approaches generally fall into
> three main categories:
>
> 1. Centralized Timestamps (Timestamp Oracle)
> 2. Atomic Clocks (True Time)
> 3. Hybrid Logical Clocks
> 4 Local Clocks

> I recommend ...<snip>... implement a pluggable time access method. This
> allows users to integrate with different time services as needed.

Huge +1

> In the mid-term, implementing the HLC approach would provide highly
> consistent snapshot reads. This offers a significant advantage for
> many use cases.

agreed

> Long-term, we should consider integrating with a distributed time
> service like AWS Time Sync Service. This ensures high accuracy and
> scalability for demanding applications.

I think the pluggable access method should make this possible, no?

--
Joe Conway
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com