Logcal replication in large scale

From: Nicklas Avén <nicklas(dot)aven(at)jordogskog(dot)no>
To: pgsql-general(at)postgresql(dot)org
Subject: Logcal replication in large scale
Date: 2017-09-20 06:53:44
Message-ID: 1505890424.29712.2.camel@jordogskog.no
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Hallo all

I am thrilled about logical replication in PostgreSQL 10. My head have
started spinning about use cases.

Would it be possible to use logical replication as a distribution
method of data?

I think about map data from national mapping authorities. The problem
is to get the updates of their data sets. Especially the open data sets
are now distributed as files (shape files) in Sweden and as pg_dump in
Norway.

I guess it is beyond what logical replication is designed for, so I ask
what problems that might arise for a scenario like:

The distributor has a publication database with logical replication
publications of the tables. All users, probably thousands or more,
would subscribe to that publication to get an updated copy of the data
set.

How would the publication server react? I guess the WAL-files will only
be written once anyway?

My guess is that it will be a lower payload than today anyway when the
whole data set have to be fetched to get updates.

In the future maybe it would be possible to make some sort of
distributed arrangement so the replication can be against any of the
client servers instead to reduce payload on the original server. But
that would of course include some way to check data integrity against
the original server with a md5 hash or something so result is identical
with the original.

ATB

Nicklas Avén

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Michael Paquier 2017-09-20 07:43:19 Re: Logical decoding client has the power to crash the server
Previous Message Meel Velliste 2017-09-20 06:14:26 Logical decoding client has the power to crash the server