Re: WALWriter active during recovery

On Tue, Dec 16, 2014 at 3:51 AM, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
> Currently, WALReceiver writes and fsyncs data it receives. Clearly,
> while we are waiting for an fsync we aren't doing any other useful
> work.
>
> Following patch starts WALWriter during recovery and makes it
> responsible for fsyncing data, allowing WALReceiver to progress other
> useful actions.

+1

> At present this is a WIP patch, for code comments only. Don't bother
> with anything other than code questions at this stage.
>
> Implementation questions are
>
> * How should we wake WALReceiver, since it waits on a poll(). Should
> we use SIGUSR1, which is already used for latch waits, or another
> signal?

Probably we need to change libpqwalreceiver so that it uses the latch.
This is useful even for the startup process to report the replay location to
the walreceiver in real time.

> * Should we introduce some pacing delays if the WALreceiver gets too
> far ahead of apply?

I don't think so for now. Instead, we can support synchronous_commit = replay,
and the users can use that new mode if they are worried about the delay of
WAL replay.

> * Other questions you may have?

Who should wake the startup process so that it reads and replays the WAL data?
Current walreceiver. But if walwriter is responsible for fsyncing WAL data,
probably walwriter should do that. Because the startup process should not replay
the WAL data which has not been fsync'd yet.

Regards,

--
Fujii Masao