Re: walprotocol.h vs frontends

On Mon, Aug 15, 2011 at 21:12, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
> On Mon, Aug 15, 2011 at 5:15 PM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
>> On Mon, Aug 15, 2011 at 18:12, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>>> Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com> writes:
>>>> Perhaps we should change the protocol so that it explicitly says which
>>>> file the streamed piece of WAL belongs to. That way a client could write
>>>> it to the correct file without knowing about all those macros.
>>>
>>> Yeah, maybe.  Otherwise, all that logic is not only implicitly part of
>>> the protocol, but essentially impossible to change because it'll be
>>> hard-coded into clients.  I wasn't too worried about this before because
>>> I supposed that only walsender and walreceiver really needed to know
>>
>> Wouldn't doing that cause significant overhead? Every single packet
>> would have to contain the filename, since the wal stream isn't
>> depending onthe filenames in where it cuts off. Also, the way it is
>> now a single packet on the replication stream can span multiple WAL
>> files... (This is the bug in my previous version that I've been able
>> to fix now)
>
> Why not have a specific protocol message to indicate a change of filename?
>
> I don't mean a WAL message, I mean a streaming protocol message.

That we could have, and it would work as long as it's always sent as
the first packet in any stream.

If we do that, we should probably make it a general metadata package -
including things like segment size as well...

> At present the WALSender only sends from one file at a time, so
> sending a message when we open a new file would be straightforward.

Are you sure? We can receive a single message spanning multiple files...

> Magnus needs some things to make this work for 9.0/9.1, but we don't
> need to change 9.2 to allow that, we just need to copy the definitions
> for those now-fixed releases.

The hack from pg_resetxlog with a manual #define FRONTEND 1 will work
for current releases, I think. And it will work for future ones as
well - but you'll be in a position where using the log streaming tool
built with different parameters (like xlog seg size) than the server
will not work - I'm not sure that's a problem big enough to worry
about, though...

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/