autovacuum: multiple workers

From: Alvaro Herrera <alvherre(at)commandprompt(dot)com>
To: Patches <pgsql-patches(at)postgresql(dot)org>
Subject: autovacuum: multiple workers
Date: 2007-03-27 21:41:22
Message-ID: 20070327214122.GA6126@alvh.no-ip.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-patches

Hi,

This is the patch to put multiple workers into autovacuum. This patch
applies after the recheck patch I just posted.

The main change is to have an array of Worker structs in shared memory;
each worker checks the current table of all other Workers, and skips a
table that's being vacuumed by any of them. It also rechecks the table
before vacuuming, which removes the problem of redundant vacuuming.

It also introduces the business of SIGUSR1 between workers and launcher.
The launcher keeps a database list in memory and schedules workers to
vacuum databases depending on that list. The actual database selected
may differ from what was in the schedule; in that case, the list is
reconstructed.

There are two main FIXMEs in this code:

1. have the list reconstruction and scheduling be smarter so that
databases are not ganged together in the schedule. The only difficulty
is keeping the sort order that the databases had.

2. have a way to clean up after failed workers filling up the Worker
array and thus starving other databases from vacuuming. I don't really
know a way to do this that works in all cases. The only idea I have so
far is that workers that started more than autovacuum_naptime seconds
ago are considered failed to start.

Neither of these is really minor, but I think they are solvable.

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

Attachment Content-Type Size
autovacuum-multiworker-4.patch text/x-diff 33.9 KB

Responses

Browse pgsql-patches by date

  From Date Subject
Next Message Simon Riggs 2007-03-27 22:33:21 Re: Patch for circular buffer in tuplestore to optimizemerge joins (v1)
Previous Message Gregory Stark 2007-03-27 21:40:13 Re: [PATCH] add CLUSTER table ORDER BY index