Re: question about postgres storage management

From: Christopher Browne <cbbrowne(at)gmail(dot)com>
To: Will Childs-Klein <willck93(at)gmail(dot)com>
Cc: PostgreSQL Mailing Lists <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: question about postgres storage management
Date: 2013-04-19 16:17:06
Message-ID: CAFNqd5UpARCQ8Ug2D6Pu5dnkU2gaq7BMzC5XyOFqpRo4i_3+kw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

I would expect the strategy you have in mind to be more useful to apply at
the filesystem level, so that it's not in Postgres altogether. (Ala
"Stacker", remember DR-DOS?)

But, to speak arguable heresy, the demerits of this sort of thing are
described nicely in Another Database's Documentation: <
http://dev.mysql.com/doc/innodb-plugin/1.0/en/innodb-compression-internals-algorithms.html
>

The relevant bit that seems to describe fairly aptly why what you are
suggesting is unlikely to turn out well:

"Some operating systems implement compression at the file system level.
Files are typically divided into fixed-size blocks that are compressed into
variable-size blocks, which easily leads into fragmentation. Every time
something inside a block is modified, the whole block is recompressed
before it is written to disk. These properties make this compression
technique unsuitable for use in an update-intensive database system."

The principle described is as applicable to Postgres as it is to InnoDB,
and is as applicable to attempting to compress disk blocks from within the
database as it is to apply it at the filesystem layer.

Postgres *does* make use of data compression, where applicable; see the
documentation for TOAST: <
http://www.postgresql.org/docs/9.2/static/storage-toast.html>

You should avail yourself of the code on TOAST:
./src/backend/catalog/toasting.c
./src/backend/access/heap/tuptoaster.c
./src/include/catalog/toasting.h
./src/include/access/tuptoaster.h

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Merlin Moncure 2013-04-19 16:32:13 Re: question about postgres storage management
Previous Message Robert Haas 2013-04-19 16:03:19 Re: [GENERAL] currval and DISCARD ALL