Re: Logical archiving

On Fri, 4 Dec 2020 at 04:33, Andrey Borodin <x4mmm(at)yandex-team(dot)ru> wrote:

>
> I was discussing problems of CDC with scientific community and they asked
> this simple question: "So you have efficient WAL archive on a very cheap
> storage, why don't you have a logical archive too?"
>

WAL archive doesn't process data; it just copies from one location into
another one. However, "logical archive" must process data.

> If we could just run archive command ```archive-tool wal-push
> 0000000900000F2C000000E1.logical``` with contents of logical replication -
> this would be super cool for OLAP. I'd prefer even avoid writing
> 0000000900000F2C000000E1.logical to disk, i.e. push data on stdio or
> something like that.
>
> The most time consuming process is logical decoding, mainly due to long
running transactions. In order to minimize your issue, we should improve
the logical decoding mechanism. There was a discussion about allowing
logical decoding on the replica that would probably help your use case a
lot.

--
Euler Taveira http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services