Re: Changing shared_buffers without restart

From: Ashutosh Bapat <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com>
To: Dmitry Dolgov <9erthalion6(at)gmail(dot)com>
Cc: Matthias van de Meent <boekewurm+postgres(at)gmail(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, pgsql-hackers(at)postgresql(dot)org, Robert Haas <robertmhaas(at)gmail(dot)com>
Subject: Re: Changing shared_buffers without restart
Date: 2025-01-13 08:11:06
Message-ID: CAExHW5tAKdTXeifgfL6zbJAzi0_H_=5ae8r5GTGg9bg8c1xuFQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi Dmitry,

On Tue, Dec 17, 2024 at 7:40 PM Ashutosh Bapat
<ashutosh(dot)bapat(dot)oss(at)gmail(dot)com> wrote:
>
> I could verify the memory mappings, their sizes etc. by looking at
> /proc/PID/maps and /proc/PID/status but I did not find a way to verify
> the amount of memory actually allocated and verify that it's actually
> shrinking and expanding. Please let me know how to verify that.

As somewhere mentioned upthread, the mmap or mremap by themselves do
not allocate any memory. Writing to the mapped region causes memory to
be allocated and shows up in VmRSS and RssShmem. But it does get
resized if mremap() shrinks the mapped region.

Attached are patches rebased on top of commit
2a7b2d97171dd39dca7cefb91008a3c84ec003ba. I have also fixed
compilation errors. Otherwise I haven't changed anything in the
patches. The last patches adds some TODOs and questions, which I think
we need to address while completing this work, just add for as a
reminder later. The TODO in postgres.c is related to your observation

> Another rough edge is that a
> backend, executing pg_reload_conf interactively, will not resize
> mappings immediately, for some reason it will require another command.
I don't have a solution right now, but at least the comment documents
the reason and points to its origin.

I am next looking at the problem of synchronizing the change across
the backends.

--
Best Wishes,
Ashutosh Bapat

Attachment Content-Type Size
0002-Allow-placing-shared-memory-mapping-with-an-20250113.patch text/x-patch 8.7 KB
0005-Use-anonymous-files-to-back-shared-memory-s-20250113.patch text/x-patch 7.2 KB
0001-Allow-to-use-multiple-shared-memory-mapping-20250113.patch text/x-patch 28.7 KB
0004-Allow-to-resize-shared-memory-without-resta-20250113.patch text/x-patch 12.5 KB
0003-Introduce-multiple-shmem-slots-for-shared-b-20250113.patch text/x-patch 10.8 KB
0006-Add-TODOs-and-questions-about-previous-comm-20250113.patch text/x-patch 10.3 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Alvaro Herrera 2025-01-13 08:44:30 Re: Psql meta-command conninfo+
Previous Message Peter Smith 2025-01-13 07:29:05 Re: Documentation update of wal_retrieve_retry_interval to mention table sync worker