Re: Protect syscache from bloating with negative cache entries

On 3/7/19 4:01 PM, Robert Haas wrote:
> On Thu, Mar 7, 2019 at 9:49 AM Tomas Vondra
> <tomas(dot)vondra(at)2ndquadrant(dot)com> wrote:
>> I don't think this shows any regression, but perhaps we should do a
>> microbenchmark isolating the syscache entirely?
>
> Well, if we need the LRU list, then yeah I think a microbenchmark
> would be a good idea to make sure we really understand what the impact
> of that is going to be. But if we don't need it and can just remove
> it then we don't.
>
>> What I had in mind is more along these lines:
>>
>> (a) track number of active syscache entries (increment when adding a new
>> one, decrement when evicting one)
>>
>> (b) track peak number of active syscache entries
>>
>> (c) after clock-sweep, if (peak > K*active) where K=2 or K=4 or so, do a
>> memory context swap, i.e. create a new context, copy active entries over
>> and destroy the old one
>>
>> That would at least free() the memory. Of course, the syscache entries
>> may have different sizes, so tracking just numbers of entries is just an
>> approximation. But I think it'd be enough.
>
> Yeah, that could be done. I'm not sure how expensive it would be, and
> I'm also not sure how much more effective it would be than what's
> currently proposed in terms of actually freeing memory. If you free
> enough dead syscache entries, you might manage to give some memory
> back to the OS: after all, there may be some locality there. And even
> if you don't, you'll at least prevent further growth, which might be
> good enough.
>

I have my doubts about that happening in practice. It might happen for
some workloads, but I think the locality is rather unpredictable.

> We could consider doing some version of what has been proposed here
> and the thing you're proposing here could later be implemented on top
> of that. I mean, evicting entries at all is a prerequisite to
> copy-and-compact.
>

Sure. I'm not saying the patch must do this to make it committable.

regards

--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services