Re: Are there plans to add data compression feature to postgresql?

On Thu, Oct 30, 2008 at 9:43 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> "Scott Marlowe" <scott(dot)marlowe(at)gmail(dot)com> writes:
>> Sure, bash Microsoft it's easy. But it doesn't address the point, is
>> a database safe on top of a compressed file system and if not, why?
>
> It is certainly *less* safe than it is on top of an uncompressed
> filesystem. Any given hardware failure will affect more stored bits
> (if the compression is effective) in a less predictable way.

Agreed. But I wasn't talking about hardware failures earlier, and
someone made the point that a compressed file system, without hardware
failure, was likely to eat your data. And I still don't think that's
true. Keep in mind a lot of the talk on this so far has been on data
warehouses, which are mostly static and well backed up. If you could
reduce the size on disk by a factor of 2 or 3, then it's worth taking
a small chance on having to recreate the whole db should something go
wrong.

To put it another way, if you find out you've got corrupted blocks in
your main db, due to bad main memory or CPU or something, are you
going to fix the bad blocks and memory and just keep going? Of course
not, you're going to reinstall from a clean backup to a clean machine.
You can"t trust the data that the machine was mangling, whether it
was on a compressed volume or not. So now your argument is one of
degree, which wasn't the discussion point I was trying to make.

> If you assume that hardware failure rates are below your level of
> concern, this doesn't matter.

I assume hardware failure rates are zero, until there is one. Then I
restore from a known good backup. compressed file systems have little
to do with that.