Re: Excessive growth of pg_attribute and other system tables

On Monday 21 March 2005 11:40 am, Tom Lane wrote:
> "Matthew T. O'Connor" <matthew(at)zeut(dot)net> writes:
> > I believe this discrepancy has to do with the fact that ANALYZE
> > can return some very bogus values for reltuples, where as vacuum
> > always returns an accurate count. I'm not sure how to best
> > handle this.
>
> I think 8.0's ANALYZE will do a better estimation job ... at least,
> Manfred Koizar rewrote the sampling algorithm in hopes of making it
> more robust.
>
> However, given that there are 9334 tuples in 82282 pages, I'd say
> that autovacuum has already failed Steve rather badly :-(. There
> shouldn't be more than a couple hundred pages given that number of
> rows. Perhaps the FSM settings are too small?

Results time. FSM settings were too small but the real problem seems
to be that pg_autovacuum isn't getting the job done.

I ran VACUUM VERBOSE and set postgresql.conf as follows (roughly 2.5
times the amounts suggested by VACUUM VERBOSE results):
max_fsm_pages = 500000
max_fsm_relations = 2500

I restarted the server last Friday night and manually did a vacuum
full and reindex of each user and system table in every database on
Monday. The zzz.pg_attribute table's file size dropped from over
600MB to less than 2MB.

It's now three days later and that table has already increased to
29MB. Processes accessing this database do create many temporary
tables so the exact count in pg_attribute varies a bit from time to
time but always hovers around 9500. I just did a manual VACUUM FULL
on pg_attribute and it's back to 1.3MB.

Upon completion of the vacuum, I restarted the pg_autovacuum daemon.
Following are the autovacuum log entries related to zzz.pg_attribute.
Note the growth of reltuples from 9532.000000 to 184720.000000 in
that time (what is a millionth of a tuple, anyway?) and the lack of
any vacuums performed.

28 12:12 PM] table name: zzz."pg_catalog"."pg_attribute"
28 12:12 PM] relid: 1249; relisshared: 0
28 12:12 PM] reltuples: 9532.000000; relpages: 157
28 12:12 PM] curr_analyze_count: 176294; curr_vacuum_count: 15447
28 12:12 PM] last_analyze_count: 176294; last_vacuum_count: 15447
28 12:12 PM] analyze_threshold: 10032; vacuum_threshold: 20064
28 12:12 PM] added table: zzz."pg_catalog"."pg_attribute"
--
28 03:42 PM] Performing: ANALYZE "pg_catalog"."pg_attribute"
28 03:42 PM] table name: zzz."pg_catalog"."pg_attribute"
28 03:42 PM] relid: 1249; relisshared: 0
28 03:42 PM] reltuples: 20270.000000; relpages: 336
28 03:42 PM] curr_analyze_count: 186374; curr_vacuum_count: 16329
28 03:42 PM] last_analyze_count: 186374; last_vacuum_count: 15447
28 03:42 PM] analyze_threshold: 20770; vacuum_threshold: 20064
--
28 10:59 PM] Performing: ANALYZE "pg_catalog"."pg_attribute"
28 10:59 PM] table name: zzz."pg_catalog"."pg_attribute"
28 10:59 PM] relid: 1249; relisshared: 0
28 10:59 PM] reltuples: 42591.000000; relpages: 706
28 10:59 PM] curr_analyze_count: 207254; curr_vacuum_count: 18156
28 10:59 PM] last_analyze_count: 207254; last_vacuum_count: 15447
28 10:59 PM] analyze_threshold: 43091; vacuum_threshold: 20064
--
29 02:03 PM] Performing: ANALYZE "pg_catalog"."pg_attribute"
29 02:03 PM] table name: zzz."pg_catalog"."pg_attribute"
29 02:03 PM] relid: 1249; relisshared: 0
29 02:03 PM] reltuples: 89464.000000; relpages: 1483
29 02:03 PM] curr_analyze_count: 250664; curr_vacuum_count: 21999
29 02:03 PM] last_analyze_count: 250664; last_vacuum_count: 15447
29 02:03 PM] analyze_threshold: 89964; vacuum_threshold: 20064
--
30 09:20 PM] Performing: ANALYZE "pg_catalog"."pg_attribute"
30 09:20 PM] table name: zzz."pg_catalog"."pg_attribute"
30 09:20 PM] relid: 1249; relisshared: 0
30 09:20 PM] reltuples: 184720.000000; relpages: 3062
30 09:20 PM] curr_analyze_count: 340791; curr_vacuum_count: 29886
30 09:20 PM] last_analyze_count: 340791; last_vacuum_count: 15447
30 09:20 PM] analyze_threshold: 185220; vacuum_threshold: 20064

Should I abandon pg_autovacuum and just do periodic VACUUM ANALYZEs of
everything? Should I use settings other than the defaults for
pg_autovacuum? Are temporary tables evil and their use to be avoided?

Just checked and zzz.pg_attribute is 50% larger than it was when I did
the VACUUM FULL at the start of this email.

Cheers,
Steve