From: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
---|---|
To: | Marc Rechté <marc4(at)rechte(dot)fr> |
Cc: | pgsql-performance(at)lists(dot)postgresql(dot)org |
Subject: | Re: OOM killer while pg_restore |
Date: | 2022-03-03 18:43:32 |
Message-ID: | 3305985.1646333012@sss.pgh.pa.us |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-performance |
=?UTF-8?Q?Marc_Recht=c3=a9?= <marc4(at)rechte(dot)fr> writes:
> Le 03/03/2022 à 16:31, Tom Lane a écrit :
>> Does memory consumption hold steady if you drop the FK constraints?
> Actually the number of rows is 232735712.
> Accordingly the RAM consumption would be x12 x3 = 7.8 GiB.
> This is close to the 8,1g I reported earlier (actually it was closer to
> 7.8 GB, due to GiB vs. GB confusion).
> So there is no memory leak.
> It took 16 hours on my box to reach that RAM consumption, and then the
> COPY failed when checking the first FK (as the referenced table was empty).
I'm guessing it was swapping like mad :-(
We've long recommended dropping FK constraints during bulk data loads,
and then re-establishing them later. That's a lot cheaper than retail
validity checks, even without the memory-consumption angle. Ideally
that sort of behavior would be automated, but nobody's gotten that
done yet. (pg_restore does do it like that during a full restore,
but not for a data-only restore, so I guess you were doing the latter.)
regards, tom lane
From | Date | Subject | |
---|---|---|---|
Next Message | aditya desai | 2022-03-04 18:31:52 | Any way to speed up INSERT INTO |
Previous Message | Marc Rechté | 2022-03-03 18:32:31 | Re: OOM killer while pg_restore |