Re: [PATCH 10/16] Introduce the concept that wal has a 'origin' node

On 20.06.2012 16:46, Simon Riggs wrote:
> The proposal now includes flag bits that would allow the addition of a
> variable length header, should that ever become necessary. So the
> unused space in the fixed header is not being "used up" as you say. In
> any case, the fixed header still has 4 wasted bytes on 64bit systems
> even after the patch is applied. So this claim of short sightedness is
> just plain wrong.
>
> ...
>
> We need to add information to every WAL record that is used as the
> source for generating LCRs. It is also possible to add this to HEAP
> and HEAP2 records, but doing that *will* bloat the WAL stream, whereas
> using the *currently wasted* bytes on a WAL record header does *not*
> bloat the WAL stream.

Or, we could provide a mechanism for resource managers to use those
padding bytes for whatever data the wish to use. Or modify the record
format so that the last 4 bytes of the data in the WAL record are always
automatically stored in those padding bytes, thus making all WAL records
4 bytes shorter. That would make the WAL even more compact, with only a
couple of extra CPU instructions in the critical path.

My point is that it's wrong to think that it's free to use those bytes,
just because they're currently unused. If we use them for one thing, we
can't use them for other things anymore. If we're so concerned about WAL
bloat that we can't afford to add any more bytes to the WAL record
header or heap WAL records, then it would be equally fruitful to look at
ways to use those padding bytes to save that precious WAL space.

I don't think we're *that* concerned about the WAL bloat, however. So
let's see what is the most sensible place to add whatever extra
information we need in the WAL, from the point of view of
maintainability, flexibility, readability etc. Then we can decide where
to put it.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com