Re: literature on write-ahead logging

On Thu, Jun 9, 2011 at 12:52 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Robert Haas <robertmhaas(at)gmail(dot)com> writes:
>> [ lots of interesting stuff about WAL optimization snipped ]
>
>> ... Second, they aren't really using locks, unless you count
>> bus locks - they appear to have implemented most or all of it via
>> CAS-based lock-free algorithms, which is probably well-justified
>> optimization effort.
>
> FWIW, I'm pretty suspicious of claims that lock-free data structures
> will be some kind of magic bullet.  As far as I can tell, a very large
> part of our contention problems on many-core machines stem from the CPUs
> fighting over cache line ownership.  Avoiding an explicit lock in favor
> of hardware primitive test-and-modify instructions isn't going to do a
> darn thing to improve that.  And contorting the algorithms until they
> fit into what's portably available in that line could easily be a net
> loss.

That's possible. It would definitely be possible to get slap-happy
with CAS, and I'm not eager to go there just because we can. On the
other hand, these lock-free algorithms seem to be springing up like
kudzu, so I doubt we'll be able to avoid them forever. People
wouldn't keep doing it if it didn't solve some problem that they have.
My suspicion is that in many cases there are better ways to optimize
that avoid the need to serialize things altogether (as the fastlock
patch does). But WAL insertion might be an exception: nobody seems to
have any good ideas about how to do that without serial locking;
unless someone does, don't see another alternative other than to
compress the critical section down to as few machine language
instructions as possible.

Anyway it's all speculation at this point: when I or someone else has
time to write some actual code, then we'll benchmark it and see what
happens.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company