Re: [PATCH] Initial progress reporting for COPY command

From: Josef Šimánek <josef(dot)simanek(at)gmail(dot)com>
To: Fujii Masao <masao(dot)fujii(at)oss(dot)nttdata(dot)com>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: [PATCH] Initial progress reporting for COPY command
Date: 2020-06-22 08:21:13
Message-ID: CAFp7QwoynYq8ZbA9+gZHeS73nDx7EzcQnUU9k4R=7tMumJ3xZw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

po 22. 6. 2020 v 4:48 odesílatel Fujii Masao <masao(dot)fujii(at)oss(dot)nttdata(dot)com>
napsal:

>
>
> On 2020/06/21 20:33, Josef Šimánek wrote:
> >
> >
> > po 15. 6. 2020 v 6:39 odesílatel Fujii Masao <
> masao(dot)fujii(at)oss(dot)nttdata(dot)com <mailto:masao(dot)fujii(at)oss(dot)nttdata(dot)com>> napsal:
> >
> >
> >
> > On 2020/06/14 21:32, Josef Šimánek wrote:
> > > Hello, as proposed by Pavel Stěhule and discussed on local czech
> PostgreSQL maillist (
> https://groups.google.com/d/msgid/postgresql-cz/CAFj8pRCZ42CBCa1bPHr7htffSV%2BNAcgcHHG0dVqOog4bsu2LFw%40mail.gmail.com?utm_medium=email&utm_source=footer)
> I have prepared an initial patch for COPY command progress reporting.
> >
> > Sounds nice!
> >
> >
> > > file - bool - is file is used?
> > > program - bool - is program used?
> >
> > Are these fields really necessary in a progress view?
> > What values are reported when STDOUT/STDIN is specified in COPY
> command?
> >
> >
> > For STDOUT and STDIN file is true and program is false.
>
> Could you tell me why these columns are necessary in *progress* view?
> If we want to see what copy command is actually running, we can see
> pg_stat_activity, instead. For example,
>
> SELECT pc.*, a.query FROM pg_stat_progress_copy pc, pg_stat_activity
> a WHERE pc.pid = a.pid;
>

If that doesn't make any sense, I can remove those. I have not strong
opinion about those values. Those were just around when I was looking for
possible values to include in the progress report.

>
> > > file_bytes_processed - amount of bytes processed when file is
> used (otherwise 0), works for both direction (
> > > FROM/TO) when file is used (file = t)
> >
> > What value is reported when STDOUT/STDIN is specified in COPY
> command?
> >
> >
> > For my first patch nothing was reported on STDOUT/STDIN usage. I'll
> attach new patch soon supporting those as well.
>
> Thanks for the patch!
>
> With the patch, pg_stat_progress_copy seems to report the progress of
> the processing on file_fdw. Is this intentional?
>

Every action using internally COPY will be included in the progress report
view.
I have spotted for example pg_dump does that and is reported there as well.
I do not see any problem regarding this. For pg_dump it is consistent with
"pg_stat_activity" reporting COPY command in the query field.

> Regards,
>
> --
> Fujii Masao
> Advanced Computing Technology Center
> Research and Development Headquarters
> NTT DATA CORPORATION
>

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Amit Kapila 2020-06-22 10:56:44 Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions
Previous Message vignesh C 2020-06-22 07:15:19 Re: [PATCH] Initial progress reporting for COPY command