Bytea/Text blob deletion is very slow...

From: Durumdara <durumdara(at)gmail(dot)com>
To: pgsql-general(at)postgresql(dot)org
Subject: Bytea/Text blob deletion is very slow...
Date: 2010-05-26 06:59:00
Message-ID: AANLkTik5yv28k5_N9fcBqCu1KO3tfUigl9fTKf00xdVF@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Hi!

PG8,4, Window XP, Python.

I have a program that makes many picture version from an original with
XNView effects.
Because I want preserve the integrity, previous datas, I used transactions,
and PGSQL.

The main problem with blobs that insertion is good, the select is good, but
the deletion is very slow.

I had 300 MB source, and 4 GB database with effect-converted images.
The table was:
blobs(id integer, file_id integer, ext char(3), size integer, blob bytea)
with primary key, and some index on file_id, ext, size.

But these indexes not matter, when I stored the blobs in the picture table
in fields, I also got this problem.

Extremely slow - this meaning that I started a query that do:

delete * from blobs where file_id in ()

(file_id have index, and select is very fast on it).

And I need to wait 2 hours for the deletion, and 1 hours for vacuum... :-(

I tried with PySQLite also. The deletion is also slow, but it was 30 minutes
only, but compact was 1 hours process...

Ok, I know that better to store blobs in other ways, but I want to preserve
the integrity, and anydbm (for example) is does not browsable, etc...

Prev. I tried with Text fields (and with Hexlify), but it was also slow. So
something is basically wrong with blobs, or blob deletion...

Thanks for your help:
dd

Browse pgsql-general by date

  From Date Subject
Next Message Jaime Casanova 2010-05-26 07:36:17 Re: Commit every N rows in PL/pgsql
Previous Message Ravi Katkar 2010-05-26 06:52:53 Re: Transaction with in function