Re: WAL insert delay settings

On Thu, Feb 21, 2019 at 12:50 PM Stephen Frost <sfrost(at)snowman(dot)net> wrote:

> > Rate limit in front of WAL insertion would allow for allocating the
> > throughput between foreground and background tasks, and even allow for
> > priority inheritance to alleviate priority inversion due to locks.
>
> I'm not sure how much we have to worry about priority inversion here as
> you need to have conflicts for that and if there's actually a conflict,
> then it seems like we should just press on.
>
> That is, a non-concurrent REINDEX is going to prevent an UPDATE from
> modifying anything in the table, which if the UPDATE is a higher
> priority than the REINDEX would be priority inversion, but that doesn't
> mean we should slow down the REINDEX to allow the UPDATE to happen
> because the UPDATE simply can't happen until the REINDEX is complete.
> Now, we might slow down the REINDEX because there's UPDATEs against
> *other* tables that aren't conflicting and we want those UPDATEs to be
> prioritized over the REINDEX but then that isn't priority inversion.
>

I was thinking along the lines that each backend gets a budget of WAL
insertion credits per time interval, and when the credits run out the
process sleeps. With this type of scheme it would be reasonably
straightforward to let UPDATEs being blocked by REINDEX to transfer their
WAL insertion budgets to the REINDEX, making it get a larger piece of the
total throughput pie.

Regards,
Ants Aasma