From: | Raj kumar <rajkumar820999(at)gmail(dot)com> |
---|---|
To: | Pgsql-admin <pgsql-admin(at)lists(dot)postgresql(dot)org> |
Subject: | Storage filled up heavily after enabling pglogical for CDC from AWS RDS to S3 |
Date: | 2021-12-12 10:48:55 |
Message-ID: | CACxU--XrvwT6Ju5=WhNtoWE=fUYTLjnuFFr=NDEc_NwzAdv7KQ@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-admin |
AWS DMS supports change data capture (CDC) using logical replication. I'm
using PostgreSQL V12 as the DMS Source and migrating data to S3 DataLake
through DMS.
*After creating replication slot for pglogical and enabling CDC from
Postgresql RDS to S3, the Storage space consumed heavily and increased by
1TB in a day and filled up the storage completely. What could be the reason
for this? How can I avoid this? Please advice.*
To enable logical replication of a self-managed PostgreSQL source database,
I followed the below steps.
1. Set wal_level = logical.
2. Set max_replication_slots to a value greater than 1.
3. Set the max_replication_slots value to 10
4. Set max_wal_senders to a value greater than 10.
5. Set wal_sender_timeout =0.
GRANT USAGE on SCHEMA pglogical to DATALAKE_USER;
GRANT ALL on ALL TABLES in SCHEMA pglogical to DATALAKE_USER;
SELECT pglogical.create_node(node_name := 'postgresdb',dsn := 'host=
postgresinstancedb.ap-south-1.rds.amazonaws.com port=5432
dbname=masterpoc');
select pglogical.create_replication_set('pgdms_replication_slot_datalake',
true, false, false, true);
Thanks & Regards,
Raj Kumar Narendiran.
From | Date | Subject | |
---|---|---|---|
Next Message | Raj kumar | 2021-12-12 10:51:21 | Re: Storage filled up heavily after enabling pglogical for CDC from AWS RDS to S3 |
Previous Message | Mladen Gogala | 2021-12-12 06:53:32 | Re: package conversion error from oracle to postgresql database !!!!!!!!! |