Re: Add on_error and log_verbosity options to file_fdw

On 2024-07-23 08:57, Michael Paquier wrote:
> On Mon, Jul 22, 2024 at 03:07:46PM -0700, Masahiko Sawada wrote:
>> I'm slightly concerned that users might not want to see the NOTICE
>> message for every scan. Unlike COPY FROM, scanning a file via file_fdw
>> could be frequent.

Agreed.

> Yeah, I also have some concerns regarding the noise that this could
> produce if called on a foreign table on a regular basis. The verbose
> mode is disabled by default so I don't see why we should not allow it
> if the relation owner wants to show it.
>
> Perhaps we should first do a silence mode for log_verbosity to skip
> the NOTICE produced at the end of the COPY FROM summarizing the whole?

I like this idea.
If there are no objections, I'm going to make a patch for this.

> It would be confusing to have different defaults between COPY and
> file_fdw, but having the option to silence that completely is also
> appealing from the user point of view.

I'm not sure we should change the defaults.
If the default of file_fdw is silence mode, I am a little concerned that
there may be cases where people think they have no errors, but in fact
they have.

>> QUERY PLAN
>> ----------------------------------------------------------------
>> Foreign Scan on public.test (cost=0.00..1.10 rows=1 width=12)
>> Output: a, b, c
>> Foreign File: test.csv
>> Foreign File Size: 12 b
>> Skipped Rows: 10
>
> Interesting idea linked to the idea of pushing the error state to
> something else than the logs. Sounds like a separate feature.

+1

--
Regards,

--
Atsushi Torikoshi
NTT DATA Group Corporation