Re: [PATCH] Initial progress reporting for COPY command

From: Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>
To: Josef Šimánek <josef(dot)simanek(at)gmail(dot)com>
Cc: pgsql-hackers(at)postgresql(dot)org, Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
Subject: Re: [PATCH] Initial progress reporting for COPY command
Date: 2020-06-15 05:34:06
Message-ID: CALj2ACUnPKtKTgL9BmvT0FjFR5UTUA46pbyRDsq0wu_AkFRh+w@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

> I'm using ftell to get current position in file to populate file_bytes_processed without error handling (ftell can return -1L and also populate errno on problems).
>
> 1. Is that a good way to get progress of file processing?

IMO, it's better to handle the error cases. One possible case where
ftell can return -1 and set errno is when the total bytes processed is
more than LONG_MAX.

Will your patch handle file_bytes_processed reporting for COPY FROM
STDIN cases? For this case, ftell can't be used.

Instead of using ftell and worrying about the errors, a simple
approach could be to have a uint64 variable in CopyStateData to track
the number of bytes read whenever CopyGetData is called. This approach
can also handle the case of COPY FROM STDIN.

With Regards,
Bharath Rupireddy.
EnterpriseDB: http://www.enterprisedb.com

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Jeff Davis 2020-06-15 06:09:55 Re: hashagg slowdown due to spill changes
Previous Message Ashutosh Bapat 2020-06-15 05:26:12 Re: [POC] Fast COPY FROM command for the table with foreign partitions