Re: reducing memory usage by using "proxy" memory contexts?

Andres Freund <andres(at)anarazel(dot)de> writes:
> I was responding to a question about postgres' per-backend memory usage,
> making me look at the various contexts below CacheMemoryContext. There
> is pretty much always a significant number of contexts below, one for
> each index:
> index info: 2048 total in 2 blocks; 568 free (1 chunks); 1480 used: pg_class_tblspc_relfilenode_index

Yup.

> But what if we had a new type of memory context that did not itself
> manage memory underlying allocations, but instead did so via the parent?
> If such a context tracked all the live allocations in some form of list,
> it could then free them from the parent at reset time. In other words,
> it'd proxy all memory management via the parent, only adding a separate
> name, and tracking of all live chunks.

I dunno, that seems like a *lot* of added overhead, and opportunity for
bugs. Maybe it'd be all right for contexts in which alloc/dealloc is
very infrequent. But why not just address this problem by reducing the
allocset blocksize parameter (some more) for these index contexts?

I'd even go a bit further, and suggest that the right way to exploit
our knowledge that these contexts' contents don't change much is to
go the other way, and reduce not increase their per-chunk overhead.
I've wanted for some time to build a context type that doesn't support
pfree() but just makes it a no-op, and doesn't round request sizes up
further than the next maxalign boundary. Without pfree we don't need
a normal chunk header; the minimum requirement of a context pointer
is enough. And since we aren't going to be recycling any chunks, there's
no need to try to standardize their sizes. This seems like it'd be ideal
for cases like the index cache contexts.

(For testing purposes, the generation.c context type might be close
enough for this, and it'd be easier to shove in.)

regards, tom lane