Re: cheaper snapshots

On Thu, Jul 28, 2011 at 3:51 AM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:

> All that having been said, even if I haven't made any severe
> conceptual errors in the above, I'm not sure how well it will work in
> practice.  On the plus side, taking a snapshot becomes O(1) rather
> than O(MaxBackends) - that's good.  On the further plus side, you can
> check both whether an XID has committed and whether it's visible to
> your snapshot in a single, atomic action with no lock - that seems
> really good.  On the minus side, checking an xid against your snapshot
> now has less locality of reference.  And, rolling over into a new
> segment of the array is going to require everyone to map it, and maybe
> cause some disk I/O as a new file gets created.

Sounds like the right set of thoughts to be having.

If you do this, you must cover subtransactions and Hot Standby. Work
in this area takes longer than you think when you take the
complexities into account, as you must.

I think you should take the premise of making snapshots O(1) and look
at all the ways of doing that. If you grab too early at a solution you
may grab the wrong one.

For example, another approach would be to use a shared hash table.
Snapshots are O(1), committing is O(k), using the snapshot is O(logN).
N can be kept small by regularly pruning the hash table. If we crash
we lose the hash table - no matter. (I'm not suggesting this is
better, just a different approach that should be judged across
others).

What I'm not sure in any of these ideas is how to derive a snapshot xmin.

--
 Simon Riggs                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services