Re: [PATCH] COPY .. COMPRESSED

* Tom Lane (tgl(at)sss(dot)pgh(dot)pa(dot)us) wrote:
> I find the argument that this supports compression-over-the-wire to be
> quite weak, because COPY is only one form of bulk data transfer, and
> one that a lot of applications don't ever use. If we think we need to
> support transmission compression for ourselves, it ought to be
> integrated at the wire protocol level, not in COPY.

As far as I can tell, COPY is the option which is strongly recommended
for bulk-data-operations. I can see the use-case for wanting SELECT
results to be compressed, but it strikes me as the 10% case for PG users
rather than the 90% one. Ditto for COPY vs. large INSERT .. VALUES.

Compressing every small packet seems like it'd be overkill and might
surprise people by actually reducing performance in the case of lots of
small requests.

It also strikes me as a bit silly to do something like:

zcat myfile.gz | \
psql -Z -c "COPY mytable FROM STDIN;"

> Just to not look like I'm rejecting stuff without proposing
> alternatives, here is an idea about a backwards-compatible design for
> doing that: we could add an option that can be set in the connection
> request packet. Say, "transmission_compression = gzip".

Alright, do we want/need to support multiple options there? What do
people think we should support? Any other particular details or issues
that come to mind with such an implementation?

I'm willing to work through that if it's the route everyone agrees with.

Thanks,

Stephen