Re: BUG #16722: PG hanging on COPY when table has close to 2^32 toasts in the table.

v4-0001-GetNewOidWithIndex_log_output.patch

* Fixed errmsg.
* Removed the notification limit and now keeps logging notification
messages.
* If the value of GETNEWOID_NOTIFY_MAX_LIMIT is exceeded, a log will be
output for each GETNEWOID_NOTIFY_MAX_LIMIT number of retries.

GETNEWOID_NOTIFY_MAX_LIMIT is set to 100 million.
On my PC, it takes about 400 seconds to retry the OID generation 100
million times.
Therefore, I think this value of GETNEWOID_NOTIFY_MAX_LIMIT is appropriate.

Best regards,

On Fri, Mar 12, 2021 at 5:23 PM tomohiro hiramitsu <hiramit(dot)tm(at)gmail(dot)com>
wrote:

>
> On Thu, Mar 11, 2021 at 5:25 PM Fujii Masao <masao(dot)fujii(at)oss(dot)nttdata(dot)com>
> wrote:
>
>>
>> On 2021/03/11 14:10, tomohiro hiramitsu wrote:
>>
>> > * v3-0001-GetNewOidWithIndex_log_output.patch
>>
>> + if (retry_count == next_notify && next_notify <=
>> GETNEWOID_NOTIFY_LIMIT)
>> + {
>> + ereport(LOG,
>> + (errmsg("failed to assign new OID in
>> relation \"%s\" after \"%llu\" retries",
>> +
>> RelationGetRelationName(relation), (unsigned long long) retry_count)));
>> + next_notify *= 2; /* double it for the next
>> notification */
>> + }
>>
>> With the patch, if the number of retries exceeds GETNEWOID_NOTIFY_LIMIT,
>> we stop logging notification messages. Why do we need to do that?
>> This behavior may cause user to get confused? For example, they may think
>> that OID has been successfully assigned because there are no notification
>> messages in the log for a while. So isn't it better to log notification
>> messages every GETNEWOID_NOTIFY_LIMIT after the number of retries
>> exceeds that?
>>
>
> Thank you for your comment.
> The purpose of setting the upper limit was to reduce the amount of log
> output.
> As you said, it may be better to keep logging to avoid confusing users.
>
> For that approach,
> GETNEWOID_NOTIFY_LIMIT needs to be renamed to
> GETNEWOID_NOTIFY_MAX_INTERVAL etc.,
> and I think it is necessary to consider how much the setting value of this
> variable should be.
>
>
>
>> Since users basically don't know the logic of GetNewOidWithIndex(),
>> they cannot understand what "XXX retries" part in the message means?
>> I have no good idea for the message. But what about something like
>> the following?
>>
>> LOG: still finding an unused OID within relation \"%s\"
>> DETAIL: \"%llu\" OID candidates were checked, but no unused OID is yet
>> found.
>>
>
> This message is more user friendly than mine.
>
> Best regards,
>