Re: Trim the heap free memory

Hi Ashutosh,

Ashutosh Bapat <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com> 于2024年8月26日周一 19:05写道：

> Hi Shawn,
> It will be good to document usage of this function. Please add
> document changes in your patch. We need to document the impact of this
> function so that users can judiciously decide whether or not to use
> this function and under what conditions. Also they would know what to
> expect when they use this function.

I have already incorporated the usage of this function into the new patch.

Currently, there is no memory information that can be extremely accurate to
reflect whether a trim operation should be performed. Here are two
conditions
that can be used as references:
1. Check the difference between the process's memory usage (for example,
the top command, due to the relationship with shared memory, it is necessary
to subtract SHR from RES) and the statistics of the memory context. If the
difference is very large, this function should be used to release memory;
2. Execute malloc_stats(). If the system bytes are greater than the
in-use bytes, this indicates that this function can be used to release
memory.

>
>
Running it after a query finishes is one thing but that can't be
> guaranteed because of the asynchronous nature of signal handlers.
> malloc_trim() may be called while a query is being executed. We need
> to assess that impact as well.
>
> Can you please share some numbers - TPS, latency etc. with and without
> this function invoked during a benchmark run?
>

I have placed malloc_trim() at the end of the exec_simple_query function,
so that malloc_trim() is executed once for each SQL statement executed. I
used pgbench to reproduce the performance impact,
and the results are as follows.
*Database preparation：*

> create database testc;
> create user t1;
> alter database testc owner to t1;
> ./pgbench testc -U t1 -i -s 100
> ./pgbench testc -U t1 -S -c 100 -j 100 -T 600

*Without Trim*:

> $./pgbench testc -U t1 -S -c 100 -j 100 -T 600
> pgbench (18devel)
> starting vacuum...end.
> transaction type: <builtin: select only>
> scaling factor: 100
> query mode: simple
> number of clients: 100
> number of threads: 100
> maximum number of tries: 1
> duration: 600 s
> number of transactions actually processed: 551984376
> number of failed transactions: 0 (0.000%)
> latency average = 0.109 ms
> initial connection time = 23.569 ms
> tps = 920001.842189 (without initial connection time)

*With Trim :*

> $./pgbench testc -U t1 -S -c 100 -j 100 -T 600
> pgbench (18devel)
> starting vacuum...end.
> transaction type: <builtin: select only>
> scaling factor: 100
> query mode: simple
> number of clients: 100
> number of threads: 100
> maximum number of tries: 1
> duration: 600 s
> number of transactions actually processed: 470690787
> number of failed transactions: 0 (0.000%)
> latency average = 0.127 ms
> initial connection time = 23.632 ms
> tps = 784511.901558 (without initial connection time)