Re: cheaper snapshots redux

Hello Dimitri,

On 08/23/2011 06:39 PM, Dimitri Fontaine wrote:
> I'm far from familiar with the detailed concepts here, but allow me to
> comment. I have two open questions:
>
> - is it possible to use a distributed algorithm to produce XIDs,
> something like Vector Clocks?
>
> Then each backend is able to create a snapshot (well, and XID) on its
> own, and any backend is still able to compare its snapshot to any
> other snapshot (well, XID)

Creation of snapshots and XID assignment are not as related as you imply
here. Keep in mind that a read-only transaction have a snapshot, but no
XID. (Not sure if it's possible for a transaction to have an XID, but
no snapshot. If it only touches system catalogs with SnapshotNow,
maybe? Don't think we support that, ATM).

> - is it possible to cache the production of the next snapshots so that
> generating an XID only means getting the next in a pre-computed
> vector?

The way I look at it, what Robert proposed can be thought of as "cache
the production of the next snapshot", with a bit of a stretch of what a
cache is, perhaps. I'd rather call it "early snapshot creation", maybe
"look-ahead something".

ATM backends all scan ProcArray to generate their snapshot. Instead,
what Robert proposes would - sometimes, somewhat - move that work from
snapshot creation time to commit time.

As Tom points out, the difficulty lies in the question of when it's
worth doing that: if you have lots of commits in a row, and no
transaction ever uses the (pre generated) snapshots of the point in time
in between, then those were wasted. OTOH, if there are just very few
COMMITs spread across lots of writes, the read-only backends will
re-create the same snapshots, over and over again. Seems wasteful as
well (as GetSnapshotData popping up high on profiles confirms somewhat).

Hope to have cleared up things a bit.

Regards

Markus