Re: CLOG extension

On Thu, May 3, 2012 at 5:59 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Robert Haas <robertmhaas(at)gmail(dot)com> writes:
>> [ CLOG extension is horrid for concurrency ]
>
> Yeah.  When that code was designed, a page's worth of transactions
> seemed like a lot so we didn't worry too much about performance glitches
> when we crossed a page boundary.  It's time to do something about it
> though.
>
> The idea of extending CLOG in advance, so that the work doesn't have to
> be done with quite so many locks held, sounds like a plan to me.  The
> one thing I'd worry about is that extension has to interact with
> freezing of very old XIDs and subsequent removal of old clog pages;
> make sure that pages will get removed before they could possibly
> get created again.
>
>> First, do we really need to WAL-log CLOG extension at all?  Perhaps
>> recovery should simply extend CLOG when it hits a commit or abort
>> record that references a page that doesn't exist yet.
>
> Maybe, but see above.  I'd be particularly worried about this in a hot
> standby situation, as you would then end up with HS queries seeing XIDs
> (in tuples) for which there was no clog page yet.  I'm inclined to think
> it's better to continue to WAL-log it, but try to arrange to do that
> without holding the other locks that are now involved.

Why not switch to 1 WAL record per file, rather than 1 per page. (32
pages, IIRC).

We can then have the whole new file written as zeroes by a background
process, which needn't do that while holding the XidGenLock.

My earlier patch to do background flushing from bgwriter can be
extended to do that.

--
 Simon Riggs                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services