Strange case of database bloat

From: Chris Travers <chris(dot)travers(at)gmail(dot)com>
To: Postgres General <pgsql-general(at)postgresql(dot)org>
Subject: Strange case of database bloat
Date: 2017-07-05 05:18:03
Message-ID: CAKt_Zfvb5_FmAotCTKC-wzqVu70ZqbvHMaGmX98nY0kZFcFzVA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Hi;

First, I haven't seen major problems of database bloat in a long time which
is why I find this case strange. I wanted to ask here what may be causing
it.

Problem:
==========
Database is in the 100GB to 200GB size range, running on btrfs (not my
choice) with nodatacow enabled (which I set up to fix a performance
issue). The workload is a very heavy batch-update workload.

The database bloats linearly. I have measured this on one table (of 149M
rows).

After vacuum full this table is (including indexes): 17GB
Every 24 hrs, seems to add its original space in size to the file system
+/-.

Bloat seems to be affecting both indexes and underlying tables.

Vacuum verbose does not indicate a disproportionate number of rows being
unremovable. So autovacuum is keeping up without too much difficulty.

Troubleshooting so far
=======================

filefrag finds a single extent on each file, so copy-on-write is not the
culprit

Selecting the smallest 10 values of ctid from one of the bloating tables
shows the first page used is around page 35 with one row per used page (and
large gaps in between).

Questions
===========
I assume that it is the fact that rows update frequently which is the
problem here? But why doesn't Postgres re-use any of the empty disk pages?

More importantly, is there anything that can be done to mitigate this issue
other than a frequent vacuum full?

--
Best Wishes,
Chris Travers

Efficito: Hosted Accounting and ERP. Robust and Flexible. No vendor
lock-in.
http://www.efficito.com/learn_more

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Jason Dusek 2017-07-05 05:22:59 Imperative Query Languages
Previous Message Tom Lane 2017-07-05 05:05:52 Re: 64bit initdb failure on macOS 10.11 and 10.12