Re: WORM and Read Only Tables (v0.1)

On 12/11/07, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:

> Compressed Tablespaces

> Using a streaming library like zlib, it will be easy to read/write data
> files into a still-usable form but with much reduced size. Access to a
> compressed table only makes sense as a SeqScan. That would be handled by
> introducing tablespace-specific access costs, discussed below. Indexes
> on compressed tables would still be allowed, but would hardly ever be
> used.

I've actually been wanting this lately, for a couple reasons. One is
reduced disk footprint, but the other is reduced I/O, similar to how
TOAST helps with large fields now. (In my particular scenario, TOAST
can't help due to small field sizes.) It would be useful to have
available even on read/write data. To that end, it would probably make
more sense to use a block compression algorithm rather than a
streaming one. Block-based algorithms can generally get better
compression than streaming ones as well, at least when fed large
enough blocks.

I'm not familiar with the implementation issues, other than the
obvious "variable block sizes make the I/O subsystem look very
different", so I don't know if there's a major tradeoff between the
two strategies (even just for read-only).

> I'm open to arguments that we don't need this at all because filesystem
> utilities exist that do everything we need. You're experience will be
> good to hear about in regard to this feature.

Some filesystems do support transparent compression, but they're not
always available. It would be nice to have compression on
unsophisticated systems with cheap hardware.