Re: Process local hint bit cache

On Wed, Mar 30, 2011 at 9:40 AM, Greg Stark <gsstark(at)mit(dot)edu> wrote:
> On Tue, Mar 29, 2011 at 10:34 PM, Merlin Moncure <mmoncure(at)gmail(dot)com> wrote:
>> So I went back to the drawing board, reviewed the archives, and came
>> up with a new proposal.  I'd like to see a process local clog page
>> cache of around 1-4 pages (8-32kb typically) that would replace the
>> current TransactionLogFetch last xid cache. It should be small,
>> because I doubt more would really help much (see below) and we want to
>> keep this data in the tight cpu caches since it's going to be
>> constantly scanned.
>>
>
> How is this different from the existing clog SLRU? It seems like the
> fundamental difference is that you plan to defer updating the hint
> bits rather than update them every time the row is accessed. That
> doesn't seem like a net win, it'll just defer the i/o, not eliminate

It is very different -- the slru layer is in shared memory and
requires locks to access. The entire point is trying to avoid
accessing this structure in tight code paths. I'm actually very
skeptical the slru layer provides any benefit at all. I bet it would
be cheaper just mmap in the pages directly (as Heikki is also
speculating).

Regarding deferring i/o, you have good chance of eliminating i/o if
the tuples are deleted or touched before a particular backend gets to
a page without already having seen the transaction (very high chance
under certain workloads). Worst case scenario, you get the old
behavior plus the overhead of maintaining the cache (which is why i'm
keen on bucketing at the page level -- so it's ultra cheap to scan and
memory efficient).

merlin