Re: cheaper snapshots

On Thu, Jul 28, 2011 at 3:46 AM, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
> Sounds like the right set of thoughts to be having.

Thanks.

> If you do this, you must cover subtransactions and Hot Standby. Work
> in this area takes longer than you think when you take the
> complexities into account, as you must.

Right. This would replace the KnownAssignedXids stuff (a non-trivial
project, I am sure).

> I think you should take the premise of making snapshots O(1) and look
> at all the ways of doing that. If you grab too early at a solution you
> may grab the wrong one.

Yeah, I'm just brainstorming at this point. This is, I think, the
best idea of what I've come up with so far, but it's definitely not
the only approach.

> For example, another approach would be to use a shared hash table.
> Snapshots are O(1), committing is O(k), using the snapshot is O(logN).
> N can be kept small by regularly pruning the hash table. If we crash
> we lose the hash table - no matter. (I'm not suggesting this is
> better, just a different approach that should be judged across
> others).

Sorry, I'm having a hard time understanding what you are describing
here. What would the keys and values in this hash table be, and what
do k and N refer to here?

> What I'm not sure in any of these ideas is how to derive a snapshot xmin.

That is a problem. If we have to scan the ProcArray every time we
take a snapshot just to derive an xmin, we are kind of hosed. One
thought I had is that we might be able to use a sort of sloppy xmin.
In other words, we keep a cached xmin, and have some heuristic where
we occasionally try to update it. A snapshot with a too-old xmin
isn't wrong, just possibly slower. But if xmin is only slightly stale
and xids can be tested relatively quickly, it might not matter very
much.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company