Re: MMAP Buffers

On Sun, 17 Apr 2011 21:06:17 -0400, Robert Haas wrote:
> On Sun, Apr 17, 2011 at 5:32 PM, Radosław Smogura
> <rsmogura(at)softperience(dot)eu> wrote:
>> Each process has simple "mirror" of shared descriptors.
>>
>> I "believe" that modifications to buffer content may be only done
>> when holding
>> exclusive lock (with some simple exceptions) (+ MVCC), actually I
>> saw only two
>> things that can change already loaded data and cause damage, you
>> have
>> described (setting hint bits during scan, and vacuum - 1st may only
>> cause, I
>> think, that two processes will ask for same transaction statuses
>> <except
>> vacuum>, 2nd one is impossible as vacumm requires exclusive pin).
>> When buffer
>> tag is changed the version of buffer is bumped up, and checked
>> against local
>> version - this about reading buffer.
>
> Yes, an exclusive lock is required for substantive content changes.
> But if vacuum cleaning up the buffer is an issue for your patch, then
> it's probably also a problem if someone grabs an exclusive content
> lock and deletes the tuple (by setting XMAX) and some other backend
> later sees the old buffer contents after having in the meanwhile
> taken
> a new snapshot; or if likewise someone grabs an exclusive-lock, adds
> a
> tuple, and then your backend takes a new snapshot and then sees the
> old buffer contents. Basically, any time someone grabs an
> exclusive-lock and releases it, it's necessary for all observers to
> see the updated contents by the time the exclusive lock is released.
>
>> In other cases after obtaining lock check is done if buffer has
>> associated
>> updatable buffer and if local "mirror" has it too, then swap should
>> take
>> place.
>
> I think this check would have to be done every time someone
> share-locks the buffer, which seems rather expensive.
>
>> Logic about updatable buffers is similar to "shared buffers", each
>> updatable
>> buffer has pin count, and updatable buffer can't be free if someone
>> uses it,
>> but in contrast to "normal buffers", updatable buffers doesn't have
>> any
>> support for locking etc. Updatable buffers exists only on free list,
>> or when
>> associated with buffer.
>
> I don't see how you're going to get away with removing buffer locks.
> They exist for a reason, and adding mmap() to the mix is going to
> require MORE locking, not less.
>
>> In future, I will change version to shared segment id, something
>> like
>> relation's oid + block, but ids will have continuous numbering
>> 1,2,3..., so I
>> will be able to bypass smgr/md during read, and tag version check -
>> this looks
>> like faster solution.
>
> I don't understand this part at all.

To my previous post I want to clarify that "updatable buffers" are
implemented in shared memory, so there is no way that process has own
copy of data.

Regards,
Radek.