Re: Scaling XLog insertion (was Re: Moving more work outside WALInsertLock)

Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com> writes:
> On 06.03.2012 17:12, Tom Lane wrote:
>> How long is the current locked code exactly --- does it contain a loop?

> Perhaps best if you take a look for yourself, the function is called
> ReserveXLogInsertLocation() in patch. It calls a helper function called
> AdvanceXLogRecPtrToNextPage(ptr), which is small and could be inlined.
> It does contain one loop, which iterates once for every WAL page the
> record crosses.

Hm. The loop makes me a tad uncomfortable, because it is possible for
WAL records to be very long (many pages). I see the point that
replacing the spinlock with an LWLock would likely negate any
performance win from this patch, but having other processes arrive and
spin while somebody is busy calculating the size of a multi-megabyte
commit record would be bad too.

What I suggest is that it should not be necessary to crawl forward one
page at a time to figure out how many pages will be needed to store N
bytes worth of WAL data. You're basically implementing a division
problem as repeated subtraction. Getting the extra WAL-segment-start
overhead right would be slightly tricky; but even if you didn't want to
try to make it pure straight-line code, at the very least it seems like
you could set it up so that the loop iterates only once per segment not
page.

regards, tom lane