Re: xlog location arithmetic

On Fri, Mar 9, 2012 at 3:04 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Robert Haas <robertmhaas(at)gmail(dot)com> writes:
>> On Fri, Mar 9, 2012 at 2:34 PM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>>> On Fri, Mar 9, 2012 at 2:26 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>>>> Hm.  I think "thousands" is an overestimate, but yeah the logic could be
>>>> greatly simplified.  However, I'm not sure we could avoid breaking the
>>>> existing naming convention for WAL files.  How much do we care about
>>>> that?
>
>>> Probably not very much, since WAL files aren't portable across major
>>> versions anyway.  But I don't see why you couldn't keep the naming
>>> convention - there's nothing to prevent you from converting a 64-bit
>>> integer back into two 32-bit integers if and where needed.
>
>> On further reflection, this seems likely to break quite a few
>> third-party tools.  Maybe it'd be worth it anyway, but it definitely
>> seems like it would be worth going to at least some minor trouble to
>> avoid it.
>
> The main actual simplification would be in getting rid of the "hole"
> at the end of each 4GB worth of WAL, cf this bit in xlog_internal.h:
>
> /*
>  * We break each logical log file (xlogid value) into segment files of the
>  * size indicated by XLOG_SEG_SIZE.  One possible segment at the end of each
>  * log file is wasted, to ensure that we don't have problems representing
>  * last-byte-position-plus-1.
>  */
> #define XLogSegSize             ((uint32) XLOG_SEG_SIZE)
> #define XLogSegsPerFile (((uint32) 0xffffffff) / XLogSegSize)
> #define XLogFileSize    (XLogSegsPerFile * XLogSegSize)
>
> If we can't get rid of that and have a continuous 64-bit WAL address
> space then it's unlikely we can actually simplify any logic.
>
> Now, doing that doesn't break the naming convention exactly; what it
> changes is that there will be WAL files numbered xxxFFFF (for some
> number of trailing-1-bits I'm too lazy to work out at the moment) where
> before there were not.  So the question really is how much external code
> there is that is aware of that specific noncontiguous numbering behavior
> and would be broken if things stopped being that way.

I would expect that most things would NOT know about that particular
foible, and just be matching pathnames on an RE, which should be fine.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company