PgBackRest and WAL archive expiry

From: KK CHN <kkchn(dot)in(at)gmail(dot)com>
To: pgsql-general <pgsql-general(at)postgresql(dot)org>
Subject: PgBackRest and WAL archive expiry
Date: 2024-09-20 05:46:10
Message-ID: CAKgGyB_TfVHh+mg-6JpPLscGO0h8_9F=km_EmQHxb69BEmsiiA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

List,

I successfully configured pgbackrest (pgBackRest 2.52.1) on RHEL 9.4 with
EPAS 16.1 for a couple of production servers and a Remote Repo Server.

Seems everything is working as expected.

I have a serious concern of archive dir growing day by day..

1. In the EPAS server I have postgres.conf with
archive_command = 'pgbackrest --stanza=EMI_Repo archive-push %p && cp %p
/data/archive/%f'

The problem is that the /data/archive folder is growing within a few
days to 850GB of 2 TB partition.

What is the mechanism to check / expire the WAL archive dir automatically?
How others control this behavior and on what criteria so that PITR won't
be badly affected if we manually delete the WALs from the archive dir ?

Does Postgres or PgBackRest have any command/directive to control the
/data/archive growth after a considerable time/usage of disk space
without affecting PITR (or on any condition ) ?

Please shed your expertise to enlighten in this regard for a healthy WAL
retention on the DB server as well as on the RepoServer

Thank you,
Krishane

For any more inputs from DB server ..

161G ./edb/as16/tablespace/ERSS
161G ./edb/as16/tablespace
167G ./edb/as16
167G ./edb
854G ./archive
229M ./backup
1.1T .
[root(at)db1 data]# cd /data/archive/
[root(at)db1 archive]# du -h
854G .
[root(at)db1 archive]#

[root(at)db1 archive]# df -h
Filesystem Size Used Avail Use% Mounted on
devtmpfs 4.0M 0 4.0M 0% /dev
tmpfs 7.7G 11M 7.7G 1% /dev/shm
tmpfs 3.1G 28M 3.1G 1% /run
/dev/mapper/rhel_bc68-root 20G 6.6G 14G 33% /
/dev/mapper/rhel_bc68-app 5.0G 68M 4.9G 2% /app

*/dev/mapper/rhel_bc68-data 2.0T 1.1T 979G 52% /data*/dev/sda2
960M 372M 589M 39% /boot
/dev/sda1 599M 7.1M 592M 2% /boot/efi
tmpfs 1.6G 52K 1.6G 1% /run/user/42
tmpfs 1.6G 36K 1.6G 1% /run/user/0
[root(at)db1 archive]#

# (change requires restart)
archive_mode = on # enables archiving; off, on, or always
# (change requires restart)
# (empty string indicates archive_command
should
# be used)
#
archive_command = 'pgbackrest --stanza=EMI_Repo archive-push %p && cp %p
/data/archive/%f'
# placeholders: %p = path of file to archive
# %f = file name only

*[root(at)db1 pg_wal]# du -h20K ./archive_status5.1G (dot)[root(at)db1
pg_wal]#*

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Christophe Pettus 2024-09-20 05:52:32 Re: PgBackRest and WAL archive expiry
Previous Message Lok P 2024-09-20 05:02:59 Re: How batch processing works