Please implement a catch-all error handler per row, for COPY

From: me nefcanto <sn(dot)1361(at)gmail(dot)com>
To: pgsql-general(at)lists(dot)postgresql(dot)org
Subject: Please implement a catch-all error handler per row, for COPY
Date: 2025-03-01 15:18:41
Message-ID: CAEHBEOCqqkSitU5gx=8aHD3ZeLtRy-1eC=Xs=xB-thFNR9JpYA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Hello

Please consider these scenarios:

- I want to create a million fake products, sometimes even 100 million
(we're on MariaDB now and we plan to migrate to Postgres). My team uses
fake data for performance tests and other use cases.
- Another scenario is translations. Even in production, we have translation
files for more than 20 languages, and for more than 2 thousand keys. That
means we need to insert 40 thousand translation records in the production.
- Another scenario is updating nested model values for a large hierarchical
table. For example, the categories table. Anytime the user changes a record
in that table we need to recalculate the nested model for the entire
categories and bulk update the results.

All of these scenarios are such that data sanitation is difficult if not
possible before doing the bulk operation (copy).

I realized that when we specify `on_error ignore` it just handles a handful
of errors. I thought this was a bug and sent an email to the pgsql-bugs
maling list. But they said it's the intended behavior.

Can you please provide a row-level catch-all handler for the copy command?

Regards
Saeed

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Tom Lane 2025-03-01 16:20:12 Re: Please implement a catch-all error handler per row, for COPY
Previous Message Krishnakant Mane 2025-03-01 05:29:20 Re: create_immv issue on aws Ubuntu even after create extention