From: | Nicklas Avén <nicklas(dot)aven(at)jordogskog(dot)no> |
---|---|
To: | pgsql-general(at)postgresql(dot)org |
Subject: | Logcal replication in large scale |
Date: | 2017-09-20 06:53:44 |
Message-ID: | 1505890424.29712.2.camel@jordogskog.no |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-general |
Hallo all
I am thrilled about logical replication in PostgreSQL 10. My head have
started spinning about use cases.
Would it be possible to use logical replication as a distribution
method of data?
I think about map data from national mapping authorities. The problem
is to get the updates of their data sets. Especially the open data sets
are now distributed as files (shape files) in Sweden and as pg_dump in
Norway.
I guess it is beyond what logical replication is designed for, so I ask
what problems that might arise for a scenario like:
The distributor has a publication database with logical replication
publications of the tables. All users, probably thousands or more,
would subscribe to that publication to get an updated copy of the data
set.
How would the publication server react? I guess the WAL-files will only
be written once anyway?
My guess is that it will be a lower payload than today anyway when the
whole data set have to be fetched to get updates.
In the future maybe it would be possible to make some sort of
distributed arrangement so the replication can be against any of the
client servers instead to reduce payload on the original server. But
that would of course include some way to check data integrity against
the original server with a md5 hash or something so result is identical
with the original.
ATB
Nicklas Avén
From | Date | Subject | |
---|---|---|---|
Next Message | Michael Paquier | 2017-09-20 07:43:19 | Re: Logical decoding client has the power to crash the server |
Previous Message | Meel Velliste | 2017-09-20 06:14:26 | Logical decoding client has the power to crash the server |