Re: Streaming replication and non-blocking I/O

Greg Stark wrote:
> On Tue, Dec 22, 2009 at 6:30 AM, Heikki Linnakangas
> <heikki(dot)linnakangas(at)enterprisedb(dot)com> wrote:
>> I think we can just use load_external_function() to load the library and
>> call WalReceiverMain from AuxiliaryProcessMain(). Ie. hard-code the
>> library name. Walreceiver is quite tightly coupled with the rest of the
>> backend anyway, so I don't think we need to come up with a pluggable API
>> at the moment.
>
> Please? I am really interested in replacing walsender and walreceiver
> with something which uses a communication bus like spread instead of a
> single point to point connection.

I think you'd still need to be able to request older WAL segments to
resync after a lost connection, restore from base backup etc., which
don't really fit into a publish/subscribe style communication bus. I'm
sure it could all be solved though. It would be a pretty cool feature,
for scaling to a large number of slaves.

> ISTM if we start with something tightly coupled it'll be hard to
> decouple later. Whereas if we start with a limited interface we'll
> learn just how much information is really required by the modules and
> will have fewer surprises later when we find suprising
> interdependencies.

I'm all ears if you have a concrete proposal.

I'm not too worried about it being hard to decouple later. The interface
is actually quite limited already, as the communication between
processes is done via shared memory. It probably wouldn't be hard to
turn it into an API, but I don't think there's a hurry to do that until
someone actually steps up to write an alternative walreceiver/walsender,

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com