Re: [PATCH] COPY .. COMPRESSED

* Tom Lane (tgl(at)sss(dot)pgh(dot)pa(dot)us) wrote:
> Really? Given that libpq provides no useful support for doing anything
> with COPY data, much less higher-level packages such as Perl DBI, I'd
> venture that the real-world ratio is more like 90/10. If not 99/1.

Perhaps I'm taking a bit too narrow view of the world, but my thinking
is OLTP won't want things compressed, as it increases latency of
requests, while OLAP users are operating with enough data that they'll
go through the effort to use COPY.

> There might be a few souls out there who are hardy enough and concerned
> enough with performance to have made their apps speak COPY protocol,
> and not given up on it the first time they hit a quoting/escaping bug
> ... but not many, I bet.

The Perl/PHP/Ruby/etc users that are writing OLTP systems aren't likely
going to be interested in this. The OLAP users are likely using psql
(it's what we're using to load terrabytes of data via COPY, JDBC, DBI,
etc, all have been tried and pale in comparison..).

> The key thing there is to force a stream buffer flush (too lazy to look
> up exactly what zlib calls it, but they have the concept) exactly when
> we're about to do a flush to the socket. That way we get cross-packet
> compression but don't have a problem with the compressor failing to send
> the last partial message when we need it to.

Yes, I'm familiar with it.

> (My suggestion of an expansible option is for future-proofing, not
> because I think we'd try to support more than one option today.)

Fair enough.

Thanks,

Stephen