Re: Process local hint bit cache

On Sat, Apr 2, 2011 at 8:37 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Merlin Moncure <mmoncure(at)gmail(dot)com> writes:
>> On Thu, Mar 31, 2011 at 5:38 PM, Merlin Moncure <mmoncure(at)gmail(dot)com> wrote:
>>> working on exanding the cache to # xid > 1.
>
>> patch attached.  this is essentially my original idea except it's
>> injected directly in to tqual.c as a kind of a expansion of the 'last
>> seen' concept.  Because the cache loops are inlined and very tight (4
>> int compares), it's actually cheaper than jumping out of line into
>> clog to test this_int = that_int;
>
> This seems like a mess.  Why is the criterion for caching commit states
> different from whether the hint bit can be set?  And why are you

The hint bits are always set. The page is flagged dirty if and only
if you have to dip below the process local cache to get it. The
rationale for this is simple: it caps your downside because unlike
hint bit removal or BM_UNTIDY, you only have to dip into clog and pay
the i/o once per page -- so that your downside is limited to pre-cache
behavior (minus the overhead of maintaining the cache).

> cluttering tqual.c with it?  Based on the initial description I'd
> expected to find this enlarging the single-entry cache in transam.c
> to have multiple entries.

This was my original idea --- abstracting it all under transam. For
simplicity's sake I was playing with a dynahash of the transaction
commit status, hit count, and log position keyed around the xid, and a
small sweeper that came around every N misses and purged out the
entries that did not have enough hit counts (or a defined minimum).

The problem with that approach at least according to my initial
testing was that putting two non-inline functions after the hint bit
test into HeapTuplesSatisifiesMVCC ate up a lot of the savings I was
looking for. I was testing on a VM, which maybe penalizes out of line
code execution more than on hardware, but I wasn't happy with it and
tried working the hint bit check angle. If you look upthread you can
see my skepticism that a non-line cache lookup is going to work.

You can trivially test this yourself if you're curious, my test was to just run:
drop table t; create table t as select v from generate_series(1,500000) v;

and repeatedly do select count(*) from v;

If you disable the hint bit action, you should see a non insignificant
difference in time even though the current last xid cache in transam
is there.

hm. If the exposure to transam is not to your liking (I'm not happy
with it either), I could make an inline stub to TransactionIdDidCommit
which checks the cache and failing anything useful there, moves to out
of line logic.

merlin