Re: Throttling WAL inserts when the standby falls behind more than the configured replica_lag_in_bytes

On Sat, Dec 25, 2021 at 6:01 PM Dilip Kumar <dilipbalaut(at)gmail(dot)com> wrote:

> On Sun, Dec 26, 2021 at 3:52 AM SATYANARAYANA NARLAPURAM <
> satyanarlapuram(at)gmail(dot)com> wrote:
>
>>
>>
>> On Fri, Dec 24, 2021 at 3:13 AM Dilip Kumar <dilipbalaut(at)gmail(dot)com>
>> wrote:
>>
>>> On Fri, Dec 24, 2021 at 3:27 AM SATYANARAYANA NARLAPURAM <
>>> satyanarlapuram(at)gmail(dot)com> wrote:
>>>
>>>>
>>>>>
>>>> XLogInsert in my opinion is the best place to call it and the hook can
>>>> be something like this "void xlog_insert_hook(NULL)" as all the throttling
>>>> logic required is the current flush position which can be obtained
>>>> from GetFlushRecPtr and the ReplicationSlotCtl. Attached a draft patch.
>>>>
>>>
>>> IMHO, it is not a good idea to call an external hook function inside a
>>> critical section. Generally, we ensure that we do not call any code path
>>> within a critical section which can throw an error and if we start calling
>>> the external hook then we lose that control.
>>>
>>
>> Thank you for the comment. XLogInsertRecord is inside a critical section
>> but not XLogInsert. Am I missing something?
>>
>
> Actually all the WAL insertions are done under a critical section (except
> few exceptions), that means if you see all the references of XLogInsert(),
> it is always called under the critical section and that is my main worry
> about hooking at XLogInsert level.
>

Got it, understood the concern. But can we document the limitations of the
hook and let the hook take care of it? I don't expect an error to be thrown
here since we are not planning to allocate memory or make file system calls
but instead look at the shared memory state and add delays when required.

>
> --
> Regards,
> Dilip Kumar
> EnterpriseDB: http://www.enterprisedb.com
>