Re: Adding new flags to XLogRecord

Why bit just add a new bitfield for flags if we need them? I'm usually
the one worried about data density so perhaps I should be on the other
side of the fence here but I'm not sure. The conventional wisdom is
that wal bandwidth is not a major issue.

greg

On 18 Sep 2008, at 12:15, Simon Riggs <simon(at)2ndQuadrant(dot)com> wrote:

>
> On Thu, 2008-09-18 at 13:56 +0300, Heikki Linnakangas wrote:
>> Simon Riggs wrote:
>>> I'd like to add some new flag bits to XLogRecord. (xlog.h)
>>>
>>> Where? xl_prev.
>>
>> I'm more curious about "What?" and "Why?", actually ;-),
>
> Just trying to solve the egg/chicken problem of "I want to add a
> flag";
> "but there are no flags available". I'm sure there's a few uses coming
> up in synch replication also.
>
> I want two flags for Hot Standby, but lets justify them on another
> post.
>
>>> So I would like to propose that we ignore the top 4 bits in
>>> xl_prev.xlogid when comparing values, rather than using all 32
>>> bits for
>>> comparison. That then frees up 4 new flag bits on XLogRecords.
>>> Changing
>>> xl_prev handling is only required in 3 places, all in xlog.c, plus
>>> some
>>> log outputs.
>>
>> Or, we could store only the delta between current record and the
>> previous one. Assuming we know what the current record is, that
>> wouldn't
>> lose any precision. That way xl_prev only needs to be as big as the
>> biggest possible WAL record we can have.
>>
>> Not that I think the precision in your scheme isn't enough, but I
>> find
>> the delta easier to comprehend.
>
> That can't work, I know, that was my first thought.
>
> The files are reused, so xl_prev protects against reusing a file and
> then having a perfectly valid WAL record *after* the correct end of
> WAL
> that makes it look like WAL continues. So a delta could be valid data
> even though the record was invalid.
>
> We don't want to zero the files cause that costs too much, so we
> have to
> allow for seemingly correct data as well as correct data in WAL.
>
> I think the xl_prev field could be removed completely when streaming,
> except the new flags of course.
>
> --
> Simon Riggs www.2ndQuadrant.com
> PostgreSQL Training, Services and Support
>
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers