Re: Trim the heap free memory

Unfortunately, I still see a compiling issue with this patch,

memtrim.c:15:10: fatal error: 'malloc.h' file not found
#include <malloc.h>
^~~~~~~~~~
1 error generated.

On Wed, 28 Aug 2024 at 12:54, shawn wang <shawn(dot)wang(dot)pg(at)gmail(dot)com> wrote:

> Hi Ashutosh,
>
> Ashutosh Bapat <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com> 于2024年8月26日周一 19:05写道：
>
>> Hi Shawn,
>> It will be good to document usage of this function. Please add
>> document changes in your patch. We need to document the impact of this
>> function so that users can judiciously decide whether or not to use
>> this function and under what conditions. Also they would know what to
>> expect when they use this function.
>
>
> I have already incorporated the usage of this function into the new patch.
>
>
> Currently, there is no memory information that can be extremely accurate to
> reflect whether a trim operation should be performed. Here are two
> conditions
> that can be used as references:
> 1. Check the difference between the process's memory usage (for example,
> the top command, due to the relationship with shared memory, it is
> necessary
> to subtract SHR from RES) and the statistics of the memory context. If the
> difference is very large, this function should be used to release memory;
> 2. Execute malloc_stats(). If the system bytes are greater than the
> in-use bytes, this indicates that this function can be used to release
> memory.
>
>>
>>
> Running it after a query finishes is one thing but that can't be
>> guaranteed because of the asynchronous nature of signal handlers.
>> malloc_trim() may be called while a query is being executed. We need
>> to assess that impact as well.
>>
>> Can you please share some numbers - TPS, latency etc. with and without
>> this function invoked during a benchmark run?
>>
>
> I have placed malloc_trim() at the end of the exec_simple_query function,
> so that malloc_trim() is executed once for each SQL statement executed. I
> used pgbench to reproduce the performance impact,
> and the results are as follows.
> *Database preparation：*
>
>> create database testc;
>> create user t1;
>> alter database testc owner to t1;
>> ./pgbench testc -U t1 -i -s 100
>> ./pgbench testc -U t1 -S -c 100 -j 100 -T 600
>
> *Without Trim*:
>
>> $./pgbench testc -U t1 -S -c 100 -j 100 -T 600
>> pgbench (18devel)
>> starting vacuum...end.
>> transaction type: <builtin: select only>
>> scaling factor: 100
>> query mode: simple
>> number of clients: 100
>> number of threads: 100
>> maximum number of tries: 1
>> duration: 600 s
>> number of transactions actually processed: 551984376
>> number of failed transactions: 0 (0.000%)
>> latency average = 0.109 ms
>> initial connection time = 23.569 ms
>> tps = 920001.842189 (without initial connection time)
>
> *With Trim :*
>
>> $./pgbench testc -U t1 -S -c 100 -j 100 -T 600
>> pgbench (18devel)
>> starting vacuum...end.
>> transaction type: <builtin: select only>
>> scaling factor: 100
>> query mode: simple
>> number of clients: 100
>> number of threads: 100
>> maximum number of tries: 1
>> duration: 600 s
>> number of transactions actually processed: 470690787
>> number of failed transactions: 0 (0.000%)
>> latency average = 0.127 ms
>> initial connection time = 23.632 ms
>> tps = 784511.901558 (without initial connection time)
>
>

--
Regards,
Rafia Sabih