Re: Cmpact commits and changeset extraction

On 2013-10-01 10:12:13 -0400, Robert Haas wrote:
> On Tue, Oct 1, 2013 at 7:26 AM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> > On 2013-10-01 06:20:20 -0400, Robert Haas wrote:
> >> On Mon, Sep 30, 2013 at 5:34 PM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> >> >> What's wrong with #1?
> >> >
> >> > It seems confusing that a changeset stream in database #1 will contain
> >> > commits (without corresponding changes) from database #2. Seems like aaa
> >> > pola violation to me.
> >>
> >> I don't really see the problem. A transaction could be empty for lots
> >> of reasons; it may have obtained an XID without writing any data, or
> >> whatever it's changed may be outside the bounds of logical rep.
> >
> > Sure. But all of them will have had a corresponding action in the
> > database. If your replication stream suddenly sees commits that you
> > cannot connect to any application activity... And it would depend on the
> > kind of commit, you won't see it if a non-compact commit was used.
> > It also means we need to do work to handle that commit. If you have a
> > busy and a less so database and you're only replicating the non-busy
> > one, that might be noticeable.
>
> Maybe. The original reason we added compact commits was because we
> thought that making unlogged tables logged and visca/versa was going
> to require adding still more stuff to the commit record. I'm no
> longer sure that's the case and, in any case, nobody's worked out the
> design details. But I can't help thinking more stuff's likely to come
> up in the future. I'm certainly willing to entertain proposals for
> restructuring that, but I don't really want to just throw it out.

Well, what I am thinking of - including & reading data depending on a
flag in ->xinfo - would give you extensibility without requiring
different types of commits. And it would only blow up the size by
whatever needs to be included.

> >> Maybe you should just skip replay of transactions with no useful
> >> content.
> >
> > Yes, I have thought about that as well. But I dislike it - how do we
> > define "no useful content"?
>
> The only action we detected for that XID was the commit itself.

What if the transaction was intentionally done to get an xid + timestamp
in a multimaster system? What if it includes DDL but no logged data? Do
we replay a transaction because of the pg_shdepend entry when creating a
table in another database?

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services