Re: Quick-and-dirty compression for WAL backup blocks

On Mon, 6 Jun 2005, Tom Lane wrote:

> Junji TERAMOTO <teramoto(dot)junji(at)lab(dot)ntt(dot)co(dot)jp> writes:
>
>> In "whole buffer log", there is a page header that includes offset of
>> "hole" (lower and upper). If we use that information, we don't need
>> any overhead, do we?
>
> No, because the WAL code cannot assume that all pages follow the
> convention that pd_lower and pd_upper represent the boundaries of
> free space. (As a counterexample: index metapages don't always
> do that.) I think the transformation has to be guaranteed lossless,
> which means that at a minimum you'd need to check whether the data
> in between pd_lower and pd_upper really is zeroes. So the irreducible
> minimum overhead is 1 bit to tell whether you compressed or not.

Vacuum doesn't zero out the free space between lower and upper, it's
just marked as unused, so a lossless compression becomes less efficient
on tables that have free space released by vacuum in them.

How about adding a flag to XLogRecData to indicate if the space between
pd_lower and pd_upper is meaningful or not? The XLogInsert caller probably
knows that. That way you could completely skip over the free space if
it's not meaningful, saving even more cycles.

- Heikki