Out of memory error handling in frontend code

From: Frédéric Yhuel <frederic(dot)yhuel(at)dalibo(dot)com>
To: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Out of memory error handling in frontend code
Date: 2023-09-28 08:14:21
Message-ID: 54153aba-821e-552a-478c-f20e33c786bd@dalibo.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hello,

One of our customers recently complained that his pg_dump stopped
abruptly with the message "out of memory".

After some time, we understood that the 20 million of large objects were
responsible for the huge memory usage (more than 10 GB) by pg_dump.

I think a more useful error message would help for such cases. Indeed,
it's not always possible to ask the client to run pg_dump with
"valgrind --tool=massif" on its server.

Now, I understand that we don't want to add too much to the frontend
code, it would be a lot of pain for not much gain.

But I wonder if we could add some checks in a few strategic places, as
in the attached patch.

The idea would be to print a more useful error message in most of the
cases, and keep the basic "out of memory" for the remaining ones.

I haven't try to get the patch ready for review, I know that the format
of the messages isn't right, I'd like to know what do you think of the
idea, first.

Best regards,
Frédéric

Attachment Content-Type Size
0001-pg_dump-fix-OOM-handling.patch text/x-patch 3.1 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Amit Langote 2023-09-28 08:26:27 Re: generic plans and "initial" pruning
Previous Message Bharath Rupireddy 2023-09-28 07:53:58 Re: [PoC] pg_upgrade: allow to upgrade publisher node