Re: Change COPY ... ON_ERROR ignore to ON_ERROR ignore_row

From: Kirill Reshke <reshkekirill(at)gmail(dot)com>
To: jian he <jian(dot)universality(at)gmail(dot)com>
Cc: Fujii Masao <masao(dot)fujii(at)oss(dot)nttdata(dot)com>, Jim Jones <jim(dot)jones(at)uni-muenster(dot)de>, "David G(dot) Johnston" <david(dot)g(dot)johnston(at)gmail(dot)com>, Yugo NAGATA <nagata(at)sraoss(dot)co(dot)jp>, torikoshia <torikoshia(at)oss(dot)nttdata(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Change COPY ... ON_ERROR ignore to ON_ERROR ignore_row
Date: 2025-01-08 07:05:01
Message-ID: CALdSSPgcuW1omVFUpnL6H63H+2qvtvEkD+4H3jUh1iEaNn_nEg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Thank you for your update.

On Fri, 13 Dec 2024 at 08:15, jian he <jian(dot)universality(at)gmail(dot)com> wrote:
>
> + /*
> + * Here we end processing of current COPY row.
> + * Update copy state counter for number of erroneous rows.
> + */
> + cstate->num_errors++;
> + cstate->escontext->error_occurred = true;
> +
> + /* Only print this NOTICE message, if it will not be followed by ERROR */
> + if (cstate->opts.log_verbosity == COPY_LOG_VERBOSITY_VERBOSE &&
> + (
> + (cstate->opts.on_error == COPY_ON_ERROR_NULL &&
> cstate->opts.reject_limit > 0 && cstate->num_errors <=
> cstate->opts.reject_limit) ||
> + (cstate->opts.on_error == COPY_ON_ERROR_IGNORE &&
> (cstate->opts.reject_limit == 0 || cstate->num_errors <=
> cstate->opts.reject_limit))
> + ))
> {
> this is kind of hard to comprehend.
> so attached is a simple version of it based on v8.

+1

So, in this version you essentially removed support for REJECT_LIMIT +
SET_TO_NULL feature? Looks like a promising change. It is more likely
to see this committed.
So, +1 on that too.

However, v9 lacks tests for REJECT_LIMIT vs erroneous rows tests.
In short, we need this message somewhere in a regression test.
```
ERROR: skipped more than REJECT_LIMIT (xxx) rows due to data type
incompatibility
```

Also, please update commit msg with all authors and reviewers. This
will make committer job a little bit easier

--
Best regards,
Kirill Reshke

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Peter Smith 2025-01-08 07:30:09 Re: POC: enable logical decoding when wal_level = 'replica' without a server restart
Previous Message vignesh C 2025-01-08 06:59:40 Re: Conflict detection for update_deleted in logical replication