vacuum'ing toast crumbs, detecting dangling transactions

From: Dave Crooke <dcrooke(at)gmail(dot)com>
To: pgsql-performance(at)postgresql(dot)org
Subject: vacuum'ing toast crumbs, detecting dangling transactions
Date: 2009-11-04 21:18:46
Message-ID: ca24673e0911041318p1c05c009l95c13b74dcb93b3@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-performance

Hi folks

I had a couple of semi-newbie questions about this, which I couldn't find
obvious answers to in the archives ... we are using Postgres 8.3, and the
behaviour is the same across Windows and Linux.

I am working with an app which, among other things stores XML files (average
about 50KB in size) in blobs in Postgres (column type "text") which Postgres
puts in a pg_toast_nnnnn table. The pattern of access is that a group of a
few hundred new rows is written to the main table once every few hours, but
then the XML documents in that recent batch of rows will be updated about
once every 5 minutes each, until the next batch of new rows is created - in
that way, the contents of the table are the most recent version of each
document, plus a historical trail of one version every few hours.

I'm not defending the decision to store blobs in a database (it was taken a
while ago, before the need for frequent updates of the XML) and it isn't
something that can be readily changed at short notice, so please no advice
about "don't do that" :-)

Obviously, the app causes high turnover of rows in both the parent table and
the toast table, so it relies heavily on vacuum to keep the size down. There
is no DBA here and no Postgres tuning has been done yet (I plan to have a
poke, but my DB tuning experience is Oracle with a side of MySQL, I am a
Postgres newbie).

Questions:

1. When I run vacuum manually on the parent table with the application
running, it has no effect on either the parent or toast table (as reported
by the "pgstattuple" add-on), even when the table is showing 40-50% dead
tuples. However, if I disconnect the app, all the dead tuples clean up and
moved to the "free space" category.

Is this a normal amount of dead space, and if not, what does this mean? My
best guess is that (a) it's not normal, and (b) somewhere the app is holding
open an old transaction, so Postgres thinks it has to retain all that data.

2. If there is a hanging transaction, what's the best way to trace it from
the PG end? Client is classic Java (Spring / Hibernate / Apache DBCP) if
that matters.

Cheers
Dave

Responses

Browse pgsql-performance by date

  From Date Subject
Next Message Kevin Grittner 2009-11-04 21:30:25 Re: vacuum'ing toast crumbs, detecting dangling transactions
Previous Message Tom Lane 2009-11-04 20:43:47 Re: maintaining a reference to a fetched row