Re: Out of memory error during updating huge table

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Gabor Siklos <siklosg(at)yahoo(dot)com>
Cc: pgsql-general(at)postgresql(dot)org
Subject: Re: Out of memory error during updating huge table
Date: 2006-02-12 00:43:33
Message-ID: 782.1139705013@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Gabor Siklos <siklosg(at)yahoo(dot)com> writes:
> I'm trying to update a table in transaction mode with
> 300 million records in it, and I'm getting an out of
> memory error.

The update per se shouldn't be a problem, but if you have AFTER ROW
triggers on the table then the list of pending trigger events could
be a problem. Foreign key constraints, in particular, use AFTER
triggers.

> In transaction mode, I first delete all the records in
> the table and then try to use COPY to populate it
> again with the new data from a file. The out of memory
> error happens during the COPY.

If the problem is coming from an FK constraint, you might consider
dropping the constraint and then re-adding it after the COPY. This'd
probably be faster than checking the rows "retail", too.

regards, tom lane

In response to

Browse pgsql-general by date

  From Date Subject
Next Message Michael Glaesemann 2006-02-12 00:44:52 Re: Each foo must have a bar
Previous Message David Fetter 2006-02-11 22:52:14 Re: Each foo must have a bar