From: | Thomas Güttler <guettliml(at)thomas-guettler(dot)de> |
---|---|
To: | pgsql-general(at)postgresql(dot)org |
Subject: | Move rows from one database to other |
Date: | 2017-02-21 08:53:31 |
Message-ID: | 8190c035-c733-0f27-530f-21fc7e629878@thomas-guettler.de |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-general |
I want to move table rows from one database to an central database.
Both run PostgreSQL.
My use case looks like this:
There are N satellite databases in different data centers. N is about 100 at the moment.
There is one central database.
I need a way to reliably move rows from the satellite databases to the central one
Example
The rows of host1 look like this:
host1, 2017-02-21, abc
host1, 2017-02-20, def
host1, 2017-02-19, ghi
The rows of host2 look like this:
host2, 2017-02-21, foo
host2, 2017-02-20, bar
host2, 2017-02-19, blu
After syncing, all lines which were transferred should be deleted on the satellite databases.
The central table should look like this (it has the same schema)
host1, 2017-02-21, abc
host1, 2017-02-20, def
host1, 2017-02-19, ghi
host2, 2017-02-21, foo
host2, 2017-02-20, bar
host2, 2017-02-19, blu
I don't want to code this myself, since there a tons of possible race conditions:
- inserts can happen during syncing.
- Network can break during syncing.
- inserts into the central table can break (e.g. disk full): No loss at the satellite database must happen.
- ...
How to solve this with PostgreSQL?
Regards,
Thomas Güttler
--
Thomas Guettler http://www.thomas-guettler.de/
From | Date | Subject | |
---|---|---|---|
Next Message | Tim Bellis | 2017-02-21 10:16:40 | Re: Autovacuum stuck for hours, blocking queries |
Previous Message | Achilleas Mantzios | 2017-02-21 07:16:05 | Re: Streaming Replication Without Downtime |