Re: Improving compressibility of WAL files

On Fri, 2009-01-09 at 01:29 +0200, Hannu Krosing wrote:
> On Thu, 2009-01-08 at 18:02 -0500, Aidan Van Dyk wrote:
...
> > There's possible a few other ways to do it, such as zero the WAL on
> > recycling (but not fsyncing it), and hopefully most of the zero's get
> > trickled out by the OS before it comes down to a single 16MB fsync, but
> > not many people seemed too enthused about the whole WAL compressablitly
> > subject...
> >
> > But, the way I see things going on -hackers, I must admit, sync-rep (WAL
> > streaming) looks like it's a long way off and possibly not even going to
> > do what I want, so *I* would really like this wal zero'ing...
> >
> > If anybody has any specific things with the patch ehty think needs
> > chaning, I'll try and accomidate, but I do note that I never
> > submitted it for the Commitfest...
>
> won't it still be easier/less intrusive on inline core functionality and
> more flexible to just record end-of-valid-wal somewhere and then let the
> compressor discard the invalid part when compressing and recreate it
> with zeros on decompression ?

And some of the functionality already exists for in-process WAL files in
form of pg_current_xlog_location() and
pg_current_xlog_insert_location(), recording end-of-data in wal file
just extends this to completed log files.

--
------------------------------------------
Hannu Krosing http://www.2ndQuadrant.com
PostgreSQL Scalability and Availability
Services, Consulting and Training