Re: storing binary data

"Jason Orendorff" <jason(at)jorendorff(dot)com> writes:

> Reply-To: sender

Just to be nice, I'll do this. ;)

> Hi. I was surprised to discover today that postgres's
> character types don't support zero bytes. That is,
> Postgres isn't 8-bit clean. Why is that?

As I understand it, the storage system itself is 8-bit clean; it's the
parser layer that isn't (as it uses C strings everywhere).

> More to the point, I need to store about 1k bytes per row
> of varying-length 8-bit binary data. I have a few options:
>
> + BLOBs. PostgreSQL BLOBs make me nervous. I worry about
> the BLOB not being deleted when the corresponding row in
> the table is deleted. The documentation is vague.

This is an issue. There is definitely no automatic deletion of
LOs. There is a 'vacuumlo' program in contrib/ that may be useful, or
you can roll your own, or you can use triggers to make sure LOs get
deleted.

FWIW, I've been using LOBs in a couple of applications and haven't had
too much trouble.

> + What I really need is a binary *short* object type.
> I have heard rumors of a legendary "bytea" type that might
> help me, but it doesn't appear to be documented anywhere,
> so I hesitate to use it.

It is in 7.1, but is more fully documented in 7.2 (which is entering
beta). See:

http://candle.pha.pa.us/main/writings/pgsql/sgml/datatype-binary.html

> + I can base64-encode the data and store it in a "text"
> field. But postgres is a great big data-storage system;
> surely it can store binary data without resorting to
> this kind of hack.

Since the only way to store or retrieve non-LOB data is to go through
the SQL parser, you always have to do some escaping. The link above
tells you how to do it for 'bytea' without having to go the base64
route.

-Doug
--
Let us cross over the river, and rest under the shade of the trees.
--T. J. Jackson, 1863