auto-vacuum conflicting with long running analyze

From: Shane Ambler <pgsql(at)Sheeky(dot)Biz>
To: PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: auto-vacuum conflicting with long running analyze
Date: 2008-04-05 18:15:46
Message-ID: 47F7C1D2.7020203@Sheeky.Biz
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

I am currently attempting to import the world street map data as
currently available from openstreetmap.org and am wondering if I will
come across a possible problem that is mentioned on their website, which
appears to be relevant for 8.1

From http://wiki.openstreetmap.org/index.php/Mapnik they state -

<quote>

After several hours the import may quit with an error like:-

ANALYZE planet_osm_line;
failed: ERROR: deadlock detected
DETAIL: Process 28511 waits for AccessExclusiveLock on relation 1064115
of database 18309; blocked by process 12776.
Process 12776 waits for ShareLock on transaction 572766655; blocked by
process 28511.

Error occurred, cleaning up

This seems to be a fault in PostgreSQL and is caused when an auto-vacuum
is attempted during the ANALYZE. The solution is to disable all
auto-vacuums on the database. The data is not updated after the import
so the vacuum process does nothing useful. In the postgresql.conf file
set the option:

autovacuum = off

Then restart the database server

# /etc/init.d/postgresql-8.1 restart

Note: In Debian/Ubuntu you also need to update
/etc/cron.d/postgresql-common to comment out the two pg_maintenance
tasks which ...

</quote>

I can see that an analyze will be run at the end of the import and given
that the uncompressed xml file being imported is about 77GB I can see a
large dataset with a long running analyze to finish of the task.

Given that the analyze will obviously take a long time, is this scenario
likely to happen with 8.3.1? or has it been fixed since 8.1.x?

Would this be the issue fixed in 8.1.1? -
Prevent autovacuum from crashing during ANALYZE of expression index

Or is it in various autovacuum improvements from 8.3.0 ?

As it is a long slow process so it will be a while before I can verify
for sure.

--

Shane Ambler
pgSQL (at) Sheeky (dot) Biz

Get Sheeky @ http://Sheeky.Biz

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Alvaro Herrera 2008-04-05 18:29:10 Re: auto-vacuum conflicting with long running analyze
Previous Message Alvaro Herrera 2008-04-05 17:25:15 Re: [HACKERS] Patch queue -> wiki