pg_restore fails when psql succeeds

From: Cherio <cherio(at)gmail(dot)com>
To: pgsql-general(at)lists(dot)postgresql(dot)org
Subject: pg_restore fails when psql succeeds
Date: 2020-12-12 20:10:05
Message-ID: CAKHqFkJykaVF8Gado-UWCu5=HDdPfihQu-X6qkNfY7jsUTDy=g@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

I am facing a consistent issue with pg_restore when moving databases with
large tables from PostgreSQL 10 to 13. pg_restore fails to restore indexes
on some large tables (anything over 20 million records).

pg_restore: error: could not execute query: ERROR: out of memory
DETAIL: Failed on request of size 214728704 in memory context "TupleSort
sort".
CONTEXT: parallel worker
Command was: CREATE INDEX some_index_idx ON schema1.table1 USING btree
(some_field);

This happens when a database is exported with formats "custom" or
"directory". No errors occur when the same databases are exported as plain
text and imported with psql.

Initially I was importing with --jobs in several threads, but reducing
threads to 1 made no difference. I tried exporting with pg_dump versions 13
and 10. It made no difference either - restore succeeds with plain text +
psql and fails with the other formats + pg_restore.

The same doesn't happen when I import from 10 into 12. I am a bit lost and
concerned at this point about moving on with conversion to version 13.

Any guidance would be greatly appreciated!

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Adrian Klaver 2020-12-12 20:18:05 Re: pg_restore fails when psql succeeds
Previous Message Alban Hertroys 2020-12-12 11:27:59 Re: SQL group by help