Re: tracking commit timestamps

On 06/11/14 01:31, Steve Singer wrote:
> On 11/05/2014 05:43 PM, Andres Freund wrote:
>
> Is this patch supposed to:
>
> A) Add commit timestamp tracking but nothing more
>
> B) Add infrastructure to store commit timestamps and provide a facility
> for storing additional bits of data extensions might want to be
> associated with the commit
>
> C). Add commit timestamps and node identifiers to commits
>
> If the answer is (A) then I can see why some people are objecting to
> adding extradata. If the answer is (B) then it's fair to ask how well
> does this patch handle various types of things people might want to
> attach to the commit record (such as the LSN). I think the problem is
> that once you start providing a facility extensions can use to store
> data along with the commit record then being restricted to 4 or 8 bytes
> is very limiting. It also doesn't allow you to load two extensions at
> once on a system. You wouldn't be able to have both the
> 'steve_commit_order' extension and BDR installed at the same time. I
> don't think this patch does a very good job at (B) but It wasn't
> intended to.
>

I would love to have (B) but I don't think that's realistic, at least
not in the extent some people on this thread would like. I mean you can
already do (B) by using table, it just isn't that great when it comes to
performance of that solution.

This patch is aimed to do limited version of (B) where you don't have
dynamic record for storing whatever you might desire but on the upside
the performance is good. And yes so far this look more like we are
actually doing (C) since main purpose of the patch is enabling conflict
detection and resolving of those conflicts, which is useful in many
replication scenarios that are not limited to the classical multi-master
setup.

> If what we are really doing is C, and just calling the space 'extradata'
> until we get the logical identifier stuff in and then we are going to
> rename extradata to nodeid then we should say so. If we are really
> concerned about the pg_upgrade impact of expanding the record later then
> maybe we should add 4 bytes of padding to the CommitTimeStampEntry now
> and but leave the manipulating the node id until later.
>

This might not be bad idea. I don't see the extradata being useful for
multiple extensions at the same time given that there is single record
per txid unless we would enforce some kind of limitation that extension
can only set the extradata for txids produced by that extension.

The namespacing idea that Andres has would again work fine for various
replication solutions as it would make it easier for them to coexist but
it would still not work for your 'steve_commit_order' (which I also
think should be done differently anyway).

In general I do see this patch to be similar in purpose to what we did
with replica triggers or logical decoding, those features also didn't
really have in-core use, were optional and enabled us to take step
forward with replication and had some side uses besides replication just
like commit timestamps do.

--
Petr Jelinek http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services