Re: Proposal for CSN based snapshots

On Mon, May 12, 2014 at 10:41 AM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> On 2014-05-12 16:56:51 +0300, Heikki Linnakangas wrote:
>> On 01/24/2014 02:10 PM, Rajeev rastogi wrote:
>> >We are also planning to implement CSN based snapshot.
>> >So I am curious to know whether any further development is happening on this.
>>
>> I started looking into this, and plan to work on this for 9.5. It's a big
>> project, so any help is welcome. The design I have in mind is to use the LSN
>> of the commit record as the CSN (as Greg Stark suggested).
>
> Cool.

Yes, very cool. I remember having some concerns about using the LSN
of the commit record as the CSN. I think the biggest one was the need
to update clog with the CSN before the commit record had been written,
which your proposal to store a temporary sentinel value there until
the commit has completed might address. However, I wonder what
happens if you write the commit record and then the attempt to update
pg_clog fails. I think you'll have to PANIC, which kind of sucks. It
would be nice to pin the pg_clog page into the SLRU before writing the
commit record so that we don't have to fear needing to re-read it
afterwards, but the SLRU machinery doesn't currently have that notion.

Another thing to think about is that LSN = CSN will make things much
more difficult if we ever want to support multiple WAL streams with a
separate LSN sequence for each. Perhaps you'll say that's a pipe
dream anyway, and I agree it's probably 5 years out, but I think it
may be something we'll want eventually. With synthetic CSNs those
systems are more decoupled. OTOH, one advantage of LSN = CSN is that
the commit order as seen on the standby would always match the commit
order as seen on the master, which is currently not true, and would be
a very nice property to have.

I think we're likely to find that system performance is quite
sensitive to any latency in updating the global-xmin. One thing about
the present system is that if you take a snapshot while a very "old"
transaction is still running, you're going to use that as your
global-xmin for the entire lifetime of your transaction. It might be
possible, with some of the rejiggering you're thinking about, to
arrange things so that there are opportunities for processes to roll
forward their notion of the global-xmin, making HOT pruning more
efficient. Whether anything good happens there or not is sort of a
side issue, but we need to make sure the efficiency of HOT pruning
doesn't regress.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company