Re: long transfer time for binary data

Andy Colson wrote:
> On 01/21/2016 03:59 PM, Johannes wrote:
>> Here are some transferring measurements (from server to client) with the
>> same file.
>>
>> scp
>> +ssl -compression 1.3 sec
>> +ssl +compression 4.6 sec
>>
>> pgadmin
>> select lo_get(12345);
>> -ssl 3.4 sec
>> +ssl +compression 5.5 sec
>> +ssl -compression 4.5 sec
>>
>> psql
>> select lo_get(12345);
>> +ssl -compression 6.0 sec
>> -ssl 4.4 sec
>>
>> java/jdbc
>> only while(in.read(buf,0,len))
>> +ssl -compression 6.0 sec
>> -ssl 3.0 sec (+ 1.8 sec for new Image())
>>
>> Here is a link for insecure ssl compression:
>> https://en.wikipedia.org/wiki/Comparison_of_TLS_implementations#Compression

> Thanks for the link on ssl compression, I'd not seen that before. I'm going to have to read up.
>
> Your numbers ... look ... odd. scp +compression is slower? pgadmin -ssl and psql -ssl and java -ssl
> are all different speeds? ssl always adds extra time? Maybe a high latency thing? If you ping the
> other box what sort of time's do you get? Maybe the extra ssl handshakes up front + high latency is
> causing it. You could try a shared/cached ssh connection to avoid the overhead.

Johannes' measurements make sense to me.

In situations where network bandwith is not the bottleneck, you will be slower with
SSL compression than without. The time you lose is the time the CPU needs to compress
and decompress the data. I observed that behaviour in one of our systems that transferred
byteas with images over SSL, which led me to introduce the "sslcompression" connection parameter
into PostgreSQL.

The variation of times between different clients could be randon (are these differences
conststent across repeated runs?), could be caused by different SSL implementations
in Java and OpenSSL or by additional processing in pgAdmin III.

Yours,
Laurenz Albe