Re: pg_basebackup compression TODO item

On Sun, Mar 6, 2016 at 7:36 PM, Euler Taveira <euler(at)timbira(dot)com(dot)br> wrote:

> On 03-03-2016 14:44, Magnus Hagander wrote:
> > On Thu, Mar 3, 2016 at 6:34 PM, Andres Freund <andres(at)anarazel(dot)de
> > <mailto:andres(at)anarazel(dot)de>> wrote:
> >
> > On 2016-03-03 18:31:03 +0100, Magnus Hagander wrote:
> > > I think we want it at protocol level rather than pg_basebackup
> level.
> >
> > I think we may want both eventually, but I do agree that protocol
> level
> > has a lot higher "priority" than that. Something like protocol level
> > compression has a bit of different tradeofs than compressing base
> > backups, and it's nice not to compress, uncompress, compress again.
> >
> >
> >
> > Yeah, good point, we definitely want both. Based on the field experience
> > I've had (which might differ from others), having it protocol level
> > would help more people tough, so should be higher prio.
> >
> Some time ago, I started a thread [1] to implement compression at
> protocol level. The use cases are data load over slow links and reduce
> bandwidth consumption during replication.
>
> At that time, there wasn't a consensus about which compression algorithm
> to choose. After the WAL compression feature, I think we can do some POC
> with LZ compression (that is already available in common).
>
> I'll try to update the code and do some benchmarks.
>
>
> +1 to protocol level compression. In our case the primary reasons why we
use thirdparty magic networking appliances as a middle man between our
offices is to compress postgres network traffic (which is very
compress-able that is > 95% reduction is normal). And the presence of
those devices introduces all kinds of weird additional error cases and
administrative overhead (+ of course cost). So I would personally consider
protocol level compression to be bigger killer feature than any other
feature that has made itself into postgres since the 9.2 release. But of
course YMMV ;-)

> [1] http://www.postgresql.org/message-id/4FD9698F.2090407@timbira.com
>
>
> --
> Euler Taveira Timbira - http://www.timbira.com.br/
> PostgreSQL: Consultoria, Desenvolvimento, Suporte 24x7 e Treinamento
>
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers
>