From: | Shane Ambler <pgsql(at)Sheeky(dot)Biz> |
---|---|
To: | PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org> |
Subject: | auto-vacuum conflicting with long running analyze |
Date: | 2008-04-05 18:15:46 |
Message-ID: | 47F7C1D2.7020203@Sheeky.Biz |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
I am currently attempting to import the world street map data as
currently available from openstreetmap.org and am wondering if I will
come across a possible problem that is mentioned on their website, which
appears to be relevant for 8.1
From http://wiki.openstreetmap.org/index.php/Mapnik they state -
<quote>
After several hours the import may quit with an error like:-
ANALYZE planet_osm_line;
failed: ERROR: deadlock detected
DETAIL: Process 28511 waits for AccessExclusiveLock on relation 1064115
of database 18309; blocked by process 12776.
Process 12776 waits for ShareLock on transaction 572766655; blocked by
process 28511.
Error occurred, cleaning up
This seems to be a fault in PostgreSQL and is caused when an auto-vacuum
is attempted during the ANALYZE. The solution is to disable all
auto-vacuums on the database. The data is not updated after the import
so the vacuum process does nothing useful. In the postgresql.conf file
set the option:
autovacuum = off
Then restart the database server
# /etc/init.d/postgresql-8.1 restart
Note: In Debian/Ubuntu you also need to update
/etc/cron.d/postgresql-common to comment out the two pg_maintenance
tasks which ...
</quote>
I can see that an analyze will be run at the end of the import and given
that the uncompressed xml file being imported is about 77GB I can see a
large dataset with a long running analyze to finish of the task.
Given that the analyze will obviously take a long time, is this scenario
likely to happen with 8.3.1? or has it been fixed since 8.1.x?
Would this be the issue fixed in 8.1.1? -
Prevent autovacuum from crashing during ANALYZE of expression index
Or is it in various autovacuum improvements from 8.3.0 ?
As it is a long slow process so it will be a while before I can verify
for sure.
--
Shane Ambler
pgSQL (at) Sheeky (dot) Biz
Get Sheeky @ http://Sheeky.Biz
From | Date | Subject | |
---|---|---|---|
Next Message | Alvaro Herrera | 2008-04-05 18:29:10 | Re: auto-vacuum conflicting with long running analyze |
Previous Message | Alvaro Herrera | 2008-04-05 17:25:15 | Re: [HACKERS] Patch queue -> wiki |