Re: TOAST compression

On Sun, Feb 26, 2006 at 09:31:05AM -0800, Luke Lonergan wrote:
> Note that this filesystem can do about 400MB/s, and we routinely see scan
> rates of 300MB/s within PG, so the real comparision is:
>
> Direct seqscan at 300MB/s versus gunzip at 77.5MB/s

So the cutover point (on your system with very fast IO) is 4:1
compression (is that 20 or 25%?). But that's assuming that PostgreSQL
can read data as fast as dd, which we all know isn't the case. That's
also assuming a pretty top-notch IO subsystem. Based on that, I'd argue
that 10% is probably a better setting, though it would be good to test
an actual case (does dbt3 produce fields large enough to ensure that
most of them will be toasted?)

Given the variables involved, maybe it makes sense to add a GUC?
--
Jim C. Nasby, Sr. Engineering Consultant jnasby(at)pervasive(dot)com
Pervasive Software http://pervasive.com work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461