Many logical replication synchronization worker threads kicking off for ONE table

From: "Dolan, Sean" <sean(dot)dolan(at)lmco(dot)com>
To: Laurenz Albe <laurenz(dot)albe(at)cybertec(dot)at>, "pgsql-general(at)lists(dot)postgresql(dot)org" <pgsql-general(at)lists(dot)postgresql(dot)org>
Subject: Many logical replication synchronization worker threads kicking off for ONE table
Date: 2023-03-16 20:38:07
Message-ID: cd6c557caa5c42af95ec1f9cf914cc8a@lmco.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

The initial email below was the end result of something "run-away" in my logical replication.

PostGres 13.6, RedHat 7.9

Database A is the publisher; Database B is the subscriber. Within Database A are multiple schemas and the publication ensures that all the schemas and its tables are added. There is one table in particular that has 1.4million rows of data.

I create a subscription on Database B and can see in the log:
LOG: logical replication table synchronization for subscription "sub_to_dbaseA", table "alert_history" has started.
CONTEXT: COPY alert_history line 6668456
LOG: logical replication table synchronization for subscription "sub_to_dbaseA", table "alert_history" has started.
CONTEXT: COPY alert_history line 5174606
LOG: logical replication table synchronization for subscription "sub_to_dbaseA", table "alert_history" has started.
CONTEXT: COPY alert_history line 4325283

Normally I would see a line for "finished", but I never do.

I then actively watch the schema/table and do \dt+ and can see that table grow in 2GB increments until I fill up the entire drive and run out of room for that schema.

I am NOT getting any "checkpoints are occurring too frequently" As I have updated my WAL size:
max_wal_size=4GB
min_wal_size=1GB

Is the system having trouble synching this amount of data in a quick fashion and therefore kicks off more synchronization threads? Anything I can do to prevent this?

Thank you

-----Original Message-----
From: Laurenz Albe <laurenz(dot)albe(at)cybertec(dot)at>
Sent: Thursday, March 16, 2023 1:25 PM
To: Dolan, Sean (US N-ISYS Technologies Inc.) <sean(dot)dolan(at)lmco(dot)com>; pgsql-general(at)lists(dot)postgresql(dot)org
Subject: EXTERNAL: Re: EXTERNAL: Re: "No Free extents", table using all allocated space but no rows!

On Thu, 2023-03-16 at 13:20 +0000, Dolan, Sean wrote:
> I messed up and confused issues.   The error is : 
> ERROR: Could not extend pg_tblspc/16555/PG_13_20200/xxxx/xxxx  No space left on device
> HINT: Check free disk space
>
> So the schema is "full" and the offender is this one table.
> I can't TRUNCATE as there needs to be space to perform the action.
> Is there a way to see if there is a transaction on that table like you allude to?

Ah, that's different.

If you don't have enough space to run TRUNCATE, and you don't feel like
extending the disk space, DROP TABLE would be a convenient alternative.

Yours,
Laurenz Albe

In response to

Browse pgsql-general by date

  From Date Subject
Next Message jian he 2023-03-17 06:47:33 src/test/examples/testlibpq2.c where the HAVE_SYS_SELECT_H is defined.
Previous Message Laurenz Albe 2023-03-16 19:24:49 Re: EXTERNAL: Re: "No Free extents", table using all allocated space but no rows!