Re: Streaming replication, and walsender during recovery

Tom Lane wrote:
> (Anyway, the argument that it's important for performance is pure
> speculation AFAIK, untainted by any actual measurements. Given the lack
> of optimization of WAL replay, it seems entirely possible that the last
> thing you want to burden a slave with is sourcing data to more slaves.)
>

On any typical production hardware, the work of WAL replay is going to
leave at least one (and probably more) CPUs idle, and have plenty of
network resources to spare too because it's just shuffling WAL in/out
rather than dealing with so many complicated client conversations. And
the thing you want to redistribute--the WAL file--is practically
guaranteed to be sitting in the OS cache at the point where you'd be
doing it, so no disk use either. You'll disrupt a little bit of
memory/CPU cache, sure, but that's about it as far as leeching resources
from the main replay in order to support the secondary slave. I'll
measure it fully the next time I have one setup to give some hard
numbers, I've never seen it rise to the point where it was worth
worrying about before to bother.

Anyway, I think what Simon was trying to suggest was that it's possible
right now to ship partial WAL files over as they advance, if you monitor
pg_xlogfile_name_offset and are willing to coordinate copying chunks
over. That basic idea is even built already--the Skytools walmgr deals
with partial WALs for example. Having all that built-into the server
with a nicer UI is awesome, but it's been possible to build something
with the same basic feature set since 8.2. Getting that going with a
chain of downstreams slaves is not so easy though, so there's something
that I think would be unique to the 9.0 implementation.

--
Greg Smith 2ndQuadrant Baltimore, MD
PostgreSQL Training, Services and Support
greg(at)2ndQuadrant(dot)com www.2ndQuadrant.com