Re: WAL format changes

On 20.06.2012 20:43, Fujii Masao wrote:
> On Wed, Jun 20, 2012 at 8:19 PM, Magnus Hagander<magnus(at)hagander(dot)net> wrote:
>> On Tue, Jun 19, 2012 at 5:57 PM, Robert Haas<robertmhaas(at)gmail(dot)com> wrote:
>>> On Tue, Jun 19, 2012 at 4:14 AM, Heikki Linnakangas
>>> <heikki(dot)linnakangas(at)enterprisedb(dot)com> wrote:
>>>> Well, that was easier than I thought. Attached is a patch to make XLogRecPtr
>>>> a uint64, on top of my other WAL format patches. I think we should go ahead
>>>> with this.
>>>
>>> +1.
>>>
>>>> The LSNs on pages are still stored in the old format, to avoid changing the
>>>> on-disk format and breaking pg_upgrade. The XLogRecPtrs stored the control
>>>> file and WAL are changed, however, so an initdb (or at least pg_resetxlog)
>>>> is required.
>>>
>>> Seems fine.
>>>
>>>> Should we keep the old representation in the replication protocol messages?
>>>> That would make it simpler to write a client that works with different
>>>> server versions (like pg_receivexlog). Or, while we're at it, perhaps we
>>>> should mandate network-byte order for all the integer and XLogRecPtr fields
>>>> in the replication protocol. That would make it easier to write a client
>>>> that works across different architectures, in>= 9.3. The contents of the
>>>> WAL would of course be architecture-dependent, but it would be nice if
>>>> pg_receivexlog and similar tools could nevertheless be
>>>> architecture-independent.
>>>
>>> I share Andres' question about how we're doing this already. I think
>>> if we're going to break this, I'd rather do it in 9.3 than 5 years
>>> from now. At this point it's just a minor annoyance, but it'll
>>> probably get worse as people write more tools that understand WAL.
>>
>> If we are looking at breaking it, and we are especially concerned
>> about something like pg_receivexlog... Is it something we could/should
>> change in the protocl *now* for 9.2, to make it non-broken in any
>> released version? As in, can we extract just the protocol change and
>> backpatch that to 9.2beta?
>
> pg_receivexlog in 9.2 cannot handle correctly the WAL location "FF"
> (which was skipped in 9.2 or before). For example, pg_receivexlog calls
> XLByteAdvance() which always skips "FF". So even if we change the protocol,
> ISTM pg_receivexlog in 9.2 cannot work well with the server in 9.3 which
> might send "FF". No?

Yeah, you can't use pg_receivexlog from 9.2 against a 9.3 server. We
can't really promise compatibility when using an older client against a
newer server, but we can try to be backwards-compatible in the other
direction. I'm thinking of using a 9.3 pg_receivexlog against a 9.2 server.

But I guess Robert is right and we shouldn't worry about
backwards-compatibility at this point. Instead, let's try to get the
protocol right, so that we can more easily provide
backwards-compatibility in the future. Like, using a 9.4 pg_receivexlog
against a 9.3 server.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com