Re: Exposing the Xact commit order to the user

On 5/26/2010 10:04 AM, Greg Stark wrote:
> Instead of discussing implementation I think you should start with the
> API the replication system needs.

... but to answer that request, actually I don't even think we should be
discussing API specifics.

During PGCon, Marco Kreen, Jim Nasby and I were discussing what the
requirements of a unified message queue, shared by Londiste and Slony
may look like. For some use cases of pgq, there isn't even any interest
in user table changes. These are simply a reliable, database backed
message passing system.

Today both systems use an "agreeable" order of changes selected by
rather expensive queries based on serializable snapshot information and
a global, non cacheable serial number.

This could be replaced with a logic based on the actual commit order of
the transactions. This order does not need to be 100% accurate. As long
as the order is recorded after all user actions have been performed
(trigger queue shut down) and while the transaction is still holding
onto its locks, that order is good enough. This will not allow a
conflicting transaction, waiting on locks to be released, to appear
having committed before the lock conflict winner.

It is obvious that in cases where only small portions or even none of
the user table changes are needed, holding on to or even parsing the
ENTIRE WAL sounds suboptimal for this use case.

Jan

--
Anyone who trades liberty for security deserves neither
liberty nor security. -- Benjamin Franklin