Re: Protect syscache from bloating with negative cache entries

At Wed, 06 Feb 2019 14:43:34 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp> wrote in <20190206(dot)144334(dot)193118280(dot)horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>
> At Tue, 5 Feb 2019 02:40:35 +0000, "Tsunakawa, Takayuki" <tsunakawa(dot)takay(at)jp(dot)fujitsu(dot)com> wrote in <0A3221C70F24FB45833433255569204D1FB93A16(at)G01JPEXMBYT05>
> > From: bruce(at)momjian(dot)us [mailto:bruce(at)momjian(dot)us]
> > > On Mon, Feb 4, 2019 at 08:23:39AM +0000, Tsunakawa, Takayuki wrote:
> > > > Horiguchi-san, Bruce, all, So, why don't we make
> > > > syscache_memory_target the upper limit on the total size of all
> > > > catcaches, and rethink the past LRU management?
> > >
> > > I was going to say that our experience with LRU has been that the
> > > overhead is not worth the value, but that was in shared resource cases,
> > > which this is not.
> >
> > That's good news! Then, let's proceed with the approach involving LRU, Horiguchi-san, Ideriha-san.
>
> If you mean accessed-time-ordered list of entries by "LRU", I
> still object to involve it since it is too complex in searching
> code paths. Invalidation would make things more complex. The
> current patch sorts entries by ct->lastaccess and discards
> entries not accessed for more than threshold, only at doubling
> cache capacity. It is already a kind of LRU in behavior.
>
> This patch intends not to let caches bloat by unnecessary
> entries, which is negative ones at first, then less-accessed ones
> currently. If you mean by "LRU" something to put a hard limit on
> the number or size of a catcache or all caches, it would be
> doable by adding sort phase before pruning, like
> CatCacheCleanOldEntriesByNum() in the attached as a PoC (first
> attched) as food for discussion.
>
> With the second attached script, we can observe what is happening
> from another session by the following query.
>
> select relname, size, ntuples, ageclass from pg_stat_syscache where relname =' pg_statistic'::regclass;
>
> > pg_statistic | 1041024 | 7109 | {{1,1109},{3,0},{30,0},{60,0},{90,6000},{0,0
>
> On the other hand, differently from the original pruning, this
> happens irrelevantly to hash resize so it will causes another
> observable intermittent slowdown than rehashing.
>
> The two should have the same extent of impact on performance when
> disabled. I'll take numbers briefly using pgbench.

Sorry, I forgot to consider references in the previous patch, and
attach the test script.

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center