Re: psql and COPY BINARY

From: Andreas Pflug <pgadmin(at)pse-consulting(dot)de>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: psql and COPY BINARY
Date: 2005-12-14 17:32:38
Message-ID: 43A05736.6090208@pse-consulting.de
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Tom Lane wrote:
> Andreas Pflug <pgadmin(at)pse-consulting(dot)de> writes:
>
>>Tom Lane wrote:
>>
>>>There wasn't any obvious bang for the buck in rewriting it.
>
>
>>Well a non-binary copy could take as much as 5 times as much as a binary
>>copy. I hit this when COPYing 1.5GB of data, getting a 6.6GB file. This
>>made the 100MBit LAN connection a bottleneck.
>
>
> Or vice versa --- the binary format is *not* necessarily smaller than text.
> As an example, an integer column that contains only small values (say 1
> or 2 digits) will need 8 bytes as binary and only 2 or 3 as text.
>
> Fixing psql to handle binary copy isn't an unreasonable thing to do,
> but I can't get real excited about it either ...

Having a choice can't be too bad.
A COMPRESSED option would be even better, but that's backend stuff (does
TOAST use an algorithm that's platform independent?). Would have reduced
the sample above to about 130MB.

Regards,
Andreas

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Zoltan Boszormenyi 2005-12-14 19:51:57 Re: Interesting speed anomaly
Previous Message Andreas Pflug 2005-12-14 17:28:04 Re: Immodest Proposal: pg_catalog.pg_ddl