Re: Review: compact fsync request queue on overflow

robertmhaas(at)gmail(dot)com (Robert Haas) writes:
> On Mon, Jan 17, 2011 at 8:23 PM, Greg Smith <greg(at)2ndquadrant(dot)com> wrote:
>> Quite.  It's taken me 12 days of machine time running pgbench to find the
>> spots where this problem occurs on a system with a reasonably sized
>> shared_buffers (I'm testing against 256MB).  It's one of those things it's
>> hard to reproduce with test data.
>>
>> Thanks for the thorough code review.  I've got a clear test plan I'm
>> progressing through this week to beat on the performance measurement aspects
>> of the patch.
>
> Any update on this? I think the test results you've posted previously
> - particularly, the fact that when the queue fills up, there are
> always many duplicates - is pretty much sufficient for us to convince
> ourselves that this will provide a benefit in cases where that occurs.

Agreed. This showed up eminently nicely when beating up the database
using pgbench.

I imagine it would be interesting to run it against a different test
than pgbench, particularly one which involves a larger number of tables.

From the behavior I have seen thus far, I'm expecting that the queue
essentially gets compressed to the size indicating the number of active
tables. With pgbench, there are 4 tables, and the queue kept getting
compressed to 3 or 4 entries that nicely corresponds with that.

> And, in cases where the queue doesn't fill up, we'll never hit the
> test that triggers this code, so it seems pretty clear there won't be
> a negative impact there either. I don't want to rush your testing
> process, but if it's already fairly clear that this will have some
> benefit, I think it would be good to get it committed and move on to
> working on the parts we're less sure about, like sorting writes and
> spreading fsyncs, where we will probably need a lot more testing than
> here to be sure that we have the right behavior.

I'm pretty happy with what I've seen thus far; I don't want to be
over-antsy about getting it all dealt with Right Quick Instantly, but it
seems like a change that doesn't have a terribly bad risk of a big
downside.
--
(reverse (concatenate 'string "ofni.secnanifxunil" "@" "enworbbc"))
The statistics on sanity are that one out of every four Americans is
suffering from some form of mental illness. Think of your three best
friends. If they're okay, then it's you. -- Rita Mae Brown