From: | Fujii Masao <masao(dot)fujii(at)oss(dot)nttdata(dot)com> |
---|---|
To: | pgsql-hackers(at)lists(dot)postgresql(dot)org |
Subject: | Re: Add new COPY option REJECT_LIMIT |
Date: | 2024-07-02 17:07:16 |
Message-ID: | fab7018f-6e00-4be4-bd1e-4a013096f6c9@oss.nttdata.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On 2024/01/26 18:49, torikoshia wrote:
> Hi,
>
> 9e2d870 enabled the COPY command to skip soft error, and I think we can add another option which specifies the maximum tolerable number of soft errors.
>
> I remember this was discussed in [1], and feel it would be useful when loading 'dirty' data but there is a limit to how dirty it can be.
>
> Attached a patch for this.
>
> What do you think?
The patch no longer applies cleanly to HEAD. Could you update it?
I think the REJECT_LIMIT feature is useful. Allowing it to be set as either the absolute number of skipped rows or a percentage of the total input rows is a good idea.
However, if we support REJECT_LIMIT, I'm not sure if the ON_ERROR option is still necessary. REJECT_LIMIT seems to cover the same cases. For instance, REJECT_LIMIT=infinity can act like ON_ERROR=ignore, and REJECT_LIMIT=0 can act like ON_ERROR=stop.
Therefore, having both ON_ERROR and REJECT_LIMIT might be confusing.
Regards,
--
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION
From | Date | Subject | |
---|---|---|---|
Next Message | Peter Geoghegan | 2024-07-02 17:09:27 | Re: Adding skip scan (including MDAM style range skip scan) to nbtree |
Previous Message | Peter Geoghegan | 2024-07-02 16:55:59 | Re: Adding skip scan (including MDAM style range skip scan) to nbtree |