Re: Latches with weak memory ordering (Re: max_wal_senders must die)

Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com> writes:
>>> Hmm, SetLatch only sets one flag, so I don't see how it could malfunction
>>> all by itself. And I would've thought that declaring the Latch variable
>>> "volatile" prevents rearrangements.
>>
>> It's not a question of code rearrangement.

Precisely. "volatile" prevents the compiler from rearranging the
instruction sequence in a way that would *issue* stores out-of-order.
However, that doesn't prevent the hardware from *executing* the stores
out-of-order from the point of view of a different processor. As Robert
noted, the risk cases here come from caching; in particular, that a
dirty cache line might get flushed to main memory later than some other
dirty cache line. There are some architectures that guarantee that this
won't happen (no doubt at significant expenditure of gates). There are
others that don't, preferring to optimize the single-processor case.
On those, you need an "msync" type of instruction to force dirty cache
lines out to main memory between any two stores whose effects had better
become visible to another processor in a certain order. I'm not sure if
it's universal, but on PPC there are actually two different sync
concepts involved, a write barrier that does the above and a read
barrier that ensures the reading processor is up-to-date.

> I believe it's safe to
> assume that two operations using a volatile pointer will not be
> rearranged wrt. each other.

This is entirely wrong, so far as cross-processor visibility of changes
is concerned. Whether it should be true in some ideal reading of the C
spec is not relevant: there are common architectures in which it is not
true.

The window for trouble is normally pretty small, and in particular any
kernel call or context swap is usually going to force an msync. So I'm
not sure that there are any risk spots in practice right now with
SetLatch. But if we expand our use of it, we can be 100% certain we'll
get bit eventually.

> Tom's other scenario, where changing some other variable in shared
> memory might not have become visible to other processes when SetLatch()
> runs, seems more plausible (if harder to run into in practice). But if
> the variable is meant to be examined by other processes, then you should
> use a lock to protect it.

In that case, of what use is the latch stuff? The whole point with that
(or at least a lot of the point) is to not have to take locks.

regards, tom lane