Seeking help extricating data from Amazon RDS Aurora/Postgres

From: Bill Mitchell <bill(at)publicrelay(dot)com>
To: General PostgreSQL List <pgsql-general(at)postgresql(dot)org>
Subject: Seeking help extricating data from Amazon RDS Aurora/Postgres
Date: 2024-01-29 19:22:10
Message-ID: a20b8943-9e37-49b6-b3de-17adf1607b69@publicrelay.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

We are attempting to extract one of our database from Amazon RDS
Aurora/Postgres to another PostgreSQL cluster that is running directly
on EC2 instances.

This particular database is currently about 3.5TB.

We are sensitive to this database being down for an extended period of
time as it will adversely impact some subsystems within our
application.  We're hoping to find a means to migrate within a 4 hour
window.

We've tried multiple approaches:

* Using Amazon DMS to copy the data
o despite doing parallelization, the total time would be around 89
hours
* Using FiveTran (great tool!) to copy data
o using this tool was that it would take about 86 *hours* to complete.
* Using a database dump and restore
o Doing a backup from RDS's replica fails, due to holding a
database lock for too long.

The ideal situation would be to have a streaming replica from Amazon
RDS, but you can only create a replica that is hosted by AWS, and have
no access to the WAL logs...

Wondering if any of the other members of this LISTSERV have tried
migrating their data off of Amazon RDS Aurora Postgres with success.

Thanks in advance for any pointers,

Bill

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Christophe Pettus 2024-01-29 19:25:31 Re: Seeking help extricating data from Amazon RDS Aurora/Postgres
Previous Message Laura Smith 2024-01-29 19:20:35 Re: Scriptable way to validate a pg_dump restore ?