Re: BUG #9161: wal_writer_delay is limited to 10s

On Mon, Feb 17, 2014 at 1:53 PM, Andres Freund <andres(at)2ndquadrant(dot)com>wrote:

> On 2014-02-17 13:48:34 -0800, Jeff Janes wrote:
> > On Fri, Feb 14, 2014 at 2:50 PM, Andres Freund <andres(at)2ndquadrant(dot)com
> >wrote:
> >
> > > On 2014-02-14 23:03:44 +0100, Clemens Eisserer wrote:
> > > > Hi Tom,
> > > >
> > > > Thanks for taking a look at my bug report.
> > > >
> > > > > But if you're generating WAL, it's going to get written anyway
> > > > > whenever a transaction commits.
> > > >
> > > > As far as I understood, this is only the case with
> > > > synchronous_commit=on, which is the reason why I turned synchrous
> > > > commit off.
> > >
> > > The WAL writer will be woken up anyway if a transaction commits
> > > (c.f. XLogSetAsyncXactLSN()).
> > >
>
> > That only happens if an 8 KB boundary has been crossed. Depending on how
> > big the records are that he is inserting, you can fit a lot of them in 8
> > KB. If the records are small, or the table us unlogged so only commit
> > messages get logged, then you could go many multiples of 10s without
> > triggering a wake up due to crossing an 8 KB boundary.
>
> Unless I miss something we wake it up if *either* a 8kb boundary has
> been crossed *or* if the async lsn hasn't been flushed yet.
>

I think it is either a 8kb boundary crossed, or the wal_writer is in "deep"
sleep (i.e. 25 times the ordinary sleep).

I don't see anything there which refers to async lsn in deciding whether to
wake.

Anyway, stracing the wal_writer shows that with a slow steady drip of small
transactions, it is definitely the wal_writer_delay which is in control,
not the latch wake up.

Also, referring to Tom's email, it does issues an fdatasync every time it
wakes up, because of this:

/* if we have already flushed that far, consider async commit records */

For this purpose, async commit could mean either commits down with
synchronous_commit=off, or commits of transactions which touched only
unlogged tables. So it writes the same 8kb block as last time (but with
new data at the logical end of the block) and flushes it.

Cheers,

Jeff