Memory Not Released After Batch Completion – Checkpointer/Background Writer Behavior , postgres 15

From: Motog Plus <mplus7535(at)gmail(dot)com>
To: pgsql-performance(at)postgresql(dot)org
Subject: Memory Not Released After Batch Completion – Checkpointer/Background Writer Behavior , postgres 15
Date: 2025-04-06 12:36:50
Message-ID: CAL5Gnit1cSHf+ux_RTvVw_2PmhHyxAk1dOQkep_um_Sc=H+18w@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-performance

Hi All,

I’m running PostgreSQL on an EC2 c5.4xlarge Ubuntu instance with the
following specs:

32 GB RAM

1.2 TB disk

16 vCPUs

Pgpool-II Configuration:

max_pool = 2

num_init_children = 1000

client_idle_limit = 300 seconds

connection_life_time = 300 seconds

load_balance_mode = on

PostgreSQL Configuration:

max_connections = 3000

checkpoint_timeout = 15min

checkpoint_completion_target = 0.9

shared_buffers = 10GB

wal_buffers = 64MB

min_wal_size = 80MB

max_wal_size = 10GB

effective_cache_size = 20GB

work_mem = 4MB

maintenance_work_mem = 1GB

bgwriter_delay = 200ms

bgwriter_lru_multiplier = 2

bgwriter_lru_maxpages = 100

max_standby_streaming_delay = 5s (on standby)

I have a primary-standby streaming replication setup, and application
modules (running in Kubernetes pods) connect to the database through
Pgpool-II using HikariCP.

---

Issue Observed:

Even when no queries are running and application batch jobs have completed,
memory usage remains high(40%)— particularly attributable to the
checkpointer and background writer processes. According to Grafana, memory
continues to remain cached.

When I manually kill the checkpointer process on the node, memory usage
drops immediately(9%). If I don't kill it and start another batch, memory
usage increases further — reaching up to 90%, though under normal
conditions it stays around 65–70%.

---

System Memory Snapshot (when idle):

free -h :
total 30 Gi
used 12 Gi
free 2.2 Gi
shared 10Gi
buff/cache 27Gi
available 18Gi
PostgreSQL memory usage from ps:

ps -eo pid,rss,cmd | grep postgres | awk '{sum+=$2} END {print sum/1024 "
MB"}'

Output:
25682.4 MB

Note: Even though the system is idle for over 50 hours, connections
connect, become idle and drop.

---

Question:

Is it normal for the checkpointer and background writer to hold onto memory
like this after workloads finish? Is this related to how PostgreSQL manages
shared buffers or dirty pages in a streaming replication setup?

Are there any parameters in postgresql.conf or Pgpool that can help ensure
memory is better reclaimed when the system is idle?

Any guidance would be much appreciated.

Best regards,
Ramzy

Responses

Browse pgsql-performance by date

  From Date Subject
Next Message Евгений Чекан 2025-04-06 14:05:18 Re: Memory Not Released After Batch Completion – Checkpointer/Background Writer Behavior , postgres 15
Previous Message Pavel Stehule 2025-04-05 10:33:21 Re: Re: proposal: schema variables