Re: Enhance file_fdw to report processed and skipped tuples in COPY progress

On Thu, Oct 3, 2024 at 6:23 AM Fujii Masao <masao(dot)fujii(at)oss(dot)nttdata(dot)com>
wrote:
>
> Hi,
>
> Currently, file_fdw updates several columns in the pg_stat_progress_copy
view,
> like relid and bytes_processed, but it doesn't track tuples_processed or
> tuples_skipped. Monitoring these would be particularly useful when
handling
> large data sets via file_fdw, as it helps track the progress of scan.
>
> The attached patch updates file_fdw to add support for reporting
> the number of tuples processed and skipped (due to on_error = 'ignore')
> in the pg_stat_progress_copy view. What are your thoughts?
>

Awesome... no failures building and it worked as expected:

-[ RECORD 1 ]----+-----------
pid | 102570
datid | 16388
datname | fabrizio
relid | 16398
command | COPY FROM
type | FILE
bytes_processed | 1359675392
bytes_total | 2921579128
tuples_processed | 18605306
tuples_excluded | 0
tuples_skipped | 0

Regards,

--
Fabrízio de Royes Mello