Re: pgbench vs. wait events

On Fri, Oct 7, 2016 at 11:34 PM, Alfred Perlstein <alfred(at)freebsd(dot)org> wrote:
>
> Are the batched writes all done before fsync is called?
>

In most cases, but if while writing, the wal segment (size - 16M) is
finished, we do fsync and then do the remaining writes and at end
again perform fsync.

> Are you sure that A only calls fsync after flushing all the buffers from B,
> C, and D? Or will it fsync twice?
>

I don't think it is guaranteed, however if B, C and D has finished
writing to wal buffers before A starts writing out buffer, then A will
ensure to combine all those writes and call fsync just once.

> Is there instrumentation to show that?
>

Currently, I don't think we have instrumentation to show that, but you
can do it if you want. In XLogFlush(), we do print the write request
values (if
XLOG_DEBUG is set and we have compiled the code with WAL_DEBUG
option), you can put the one print in XLOGWrite once the write/flush
is done and another print when the request is satisfied without
calling XLOGWrite.

> I know there's a tremendous level of skill involved in this code, but simply
> asking in case there's some tricks.
>
> Another strategy that may work is actually intentionally waiting/buffering
> some few ms between flushes/fsync,

We do that before attempting to write if user has set "commit_delay"
and "commit_siblings" guc parameters.

Now here, we can't buffer the fsync requests as current we are doing
both writes and fsync under one lock. However, if we can split the
work such that writes are done under one lock and fsync under separate
lock, then probably we can try to buffer fsync requests and after
fsyncing the current pending requests, we can recheck if there are
more pending requests and try to flush them.

--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com