Re: xlog.c: WALInsertLock vs. WALWriteLock

On Mon, Oct 25, 2010 at 6:31 PM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> On Fri, Oct 22, 2010 at 3:08 PM, fazool mein <fazoolmein(at)gmail(dot)com> wrote:
>> I'm writing a function that will read data from the buffer in xlog (i.e.
>> from XLogCtl->pages and XLogCtl->xlblocks). I want to make sure that I am
>> doing it correctly.
>> For reading from the buffer, do I need to lock WALInsertLock or
>> WALWriteLock? Also, can you explain a bit the usage of 'LW_SHARED'. Can we
>> use it for read purposes?
>
> Holding WALInsertLock in shared mode prevents other processes from
> inserting WAL, or in other words it keeps the "end" position from
> moving, while holding WALWriteLock in shared mode prevents other
> processes from writing the WAL from the buffers out to the operating
> system, or in other words it keeps the "start" position from moving.
> So you could probably take WALInsertLock in shared mode, figure out
> the current end of WAL position, release the lock;

Once you release the WALInsertLock, someone else can grab it and
overwrite the part of the buffer you think you are reading.
So I think you have to hold WALInsertLock throughout the duration of
the operation.

Of course it couldn't be overwritten if the wal record itself is not
yet written from buffer to the OS/disk. But since you are not yet
holding the WALWriteLock, this could be happening at any time.

> then take
> WALWriteLock in shared mode, read any WAL before the end of WAL
> position, and release the lock.  But note that this wouldn't guarantee
> that you read all WAL as it's generated....

I don't think that holding WALWriteLock accomplishes much. It
prevents part of the buffer from being written out to OS/disk, and
thus becoming eligible for being overwritten in the buffer, but the
WALInsertLock prevents it from actually being overwritten. And what
if the part of the buffer you want to read was already eligible for
overwriting but not yet actually overwritten? WALWriteLock won't
allow you to safely access it, but WALInsertLock will (assuming you
have a safe way to identify the record in the first place). For
either case, holding it in shared mode would be sufficient.

Jeff