Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

Hi,

> save the error metadata to system catalogs would be more expensive,
> please see below explanation.
> I have no knowledge of publications.
> but i feel there is a feature request: publication FOR ALL TABLES
> exclude regex_pattern.
> Anyway, that would be another topic.

I think saving error metadata to system catalog is not a good idea, too.
And I believe Sawada-san just pointed out missing features and did not
suggested that we use system catalog.

> I don't think "specify the maximum number of errors to tolerate
> before raising an ERROR." is very useful....

That may be so.
I imagine it's useful in some use case since some loading tools have
such options.
Anyway I agree it's not necessary for initial patch as mentioned in [1].

> I suppose we can specify an ERRORFILE directory. similar
> implementation [2], demo in [3]
> it will generate 2 files, one file shows the malform line content as
> is, another file shows the error info.

That may be a good option when considering "(2) logging errors to
somewhere".

What do you think about the proposal to develop these features in
incrementally?

On 2023-12-15 05:48, Masahiko Sawada wrote:
> So I'm thinking we may be able to implement this
> feature incrementally. The first step would be something like an
> option to ignore all errors or an option to specify the maximum number
> of errors to tolerate before raising an ERROR. The second step would
> be to support logging destinations such as server logs and tables.

[1]
https://www.postgresql.org/message-id/752672.1699474336%40sss.pgh.pa.us

--
Regards,

--
Atsushi Torikoshi
NTT DATA Group Corporation