Re: Enhancing Memory Context Statistics Reporting

Hi,

To achieve both completeness and avoid writing to a file, I can consider
> displaying the numbers for the remaining contexts as a cumulative total
> at the end of the output.
>
> Something like follows:
> ```
> postgres=# select * from pg_get_process_memory_contexts('237244', false);
> name | ident
> | type | path | total_bytes | tot
> al_nblocks | free_bytes | free_chunks | used_bytes | pid
>
> ---------------------------------------+------------------------------------------------+----------+--------------+-------------+----
> -----------+------------+-------------+------------+--------
> TopMemoryContext |
> | AllocSet | {0} | 97696 |
> 5 | 14288 | 11 | 83408 | 237244
> search_path processing cache |
> | AllocSet | {0,1} | 8192 |
> 1 | 5328 | 7 | 2864 | 237244
>
> *Remaining contexts total: 23456 bytes (total_bytes) ,
> 12345(used_bytes), 11,111(free_bytes)*
> ```
>

Please find attached an updated patch with this change. The file previously
used to
store spilled statistics has been removed. Instead, a cumulative total of
the
remaining/spilled context statistics is now stored in the DSM segment,
which is
displayed as follows.

postgres=# select * from pg_get_process_memory_contexts('352966', false);
* name * | ident | type | path | *total_bytes*
| *total_nblocks* | *free_bytes *| *free_chunks *| *used_bytes* | pi
d
------------------------------+-------+----------+--------+-------------+---------------+------------+-------------+------------+----
----
TopMemoryContext | | AllocSet | {0} | 97696 |
5 | 14288 | 11 | 83408 | 352
966
.
.
.
MdSmgr | | AllocSet | {0,18} | 8192 |
1 | 7424 | 0 | 768 | 352
966
* Remaining Totals* | | | | *1756016*
| *188 *| *658584 *| * 132* | * 1097432 *| 352
966
(7129 rows)
-----

I believe this serves as a good compromise between completeness
and avoiding the overhead of file handling. However, I am open to
reintroducing file handling if displaying the complete statistics of the
remaining contexts prove to be more important.

All the known bugs in the patch have been fixed.

In summary, one DSA per PostgreSQL process is used to share
the statistics of that process. A DSA is created by the first client
backend that requests memory context statistics, and it is pinned
for all future requests to that process.
A handle to this DSA is shared between the client and the publishing
process using fixed shared memory. The fixed shared memory consists
of an array of size MaxBackends + auxiliary processes, indexed
by procno. Each element in this array is less than 100 bytes in size.

A PostgreSQL process uses a condition variable to signal a waiting client
backend once it has finished publishing the statistics. If, for some
reason,
the signal is not sent, the waiting client backend will time out.

When statistics of a local backend is requested, this function returns the
following
WARNING and exits, since this can be handled by an existing function which
doesn't require a DSA.

WARNING: cannot return statistics for local backend
HINT: Use pg_get_backend_memory_contexts instead

Looking forward to your review.

Thank you,
Rahila Syed