Re: Reducing memory consumption for pending inval messages

On 5/30/21, 10:22 AM, "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> We can do a lot better, by exploiting what we know about the usage
> patterns of invalidation requests. New requests are always added to
> the latest sublist, and the only management actions are (1) merge
> latest sublist into next-to-latest sublist, or (2) drop latest
> sublist, if a subtransaction aborts. This means we could perfectly
> well keep all the requests in a single, densely packed array in
> TopTransactionContext, and replace the "list" control structures
> with indexes into that array. The attached patch does that.

I spent some time looking through this patch, and it seems reasonable
to me.

> There is one notable new assumption I had to make for this. At end
> of a subtransaction, we have to merge its inval events into the
> "PriorCmd" list of the parent subtransaction. (It has to be the
> PriorCmd list, not the CurrentCmd list, because these events have
> already been processed locally; we don't want to do that again.)
> This means the parent's CurrentCmd list has to be empty at that
> instant, else we'd be trying to merge sublists that aren't adjacent
> in the array. As far as I can tell, this is always true: the patch's
> check for it doesn't trigger in a check-world run. And there's an
> argument that it must be true for semantic consistency (see comments
> in patch). So if that check ever fails, it probably means there is a
> missing CommandCounterIncrement somewhere. Still, this could use more
> review and testing.

I didn't discover any problems with this assumption in my testing,
either. Perhaps it'd be good to commit something like this sooner in
the v15 development cycle to maximize the amount of coverage it gets.

Nathan