Re: Proposal: custom compression methods

On 12/14/15 12:50 AM, Craig Ringer wrote:
> The issue with per-Datum is that TOAST claims two bits of a varlena
> header, which already limits us to 1 GiB varlena values, something
> people are starting to find to be a problem. There's no wiggle room to
> steal more bits. If you want pluggable compression you need a way to
> store knowledge of how a given datum is compressed with the datum or
> have a fast, efficient way to check.

... unless we allowed for 8 byte varlena headers. Compression changes
themselves certainly don't warrant that, but if people are already
unhappy with 1GB TOAST then maybe that's enough.

The other thing this might buy us are a few bits that could be used to
support Datum versioning for other purposes, such as when the binary
format of something changes. I would think that at some point we'll need
that for pg_upgrade.
--
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com