From: | Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> |
---|---|
To: | John Naylor <john(dot)naylor(at)enterprisedb(dot)com> |
Cc: | Nathan Bossart <nathandbossart(at)gmail(dot)com>, Andres Freund <andres(at)anarazel(dot)de>, Matthias van de Meent <boekewurm+postgres(at)gmail(dot)com>, Yura Sokolov <y(dot)sokolov(at)postgrespro(dot)ru>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: [PoC] Improve dead tuple storage for lazy vacuum |
Date: | 2023-03-09 06:51:05 |
Message-ID: | CAD21AoC+kjqkSnLLcbqtu=Td5HSx2H+iAwNHCeZcWqjy1w7k5g@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On Wed, Mar 8, 2023 at 1:40 PM John Naylor <john(dot)naylor(at)enterprisedb(dot)com> wrote:
>
>
> On Tue, Mar 7, 2023 at 8:25 AM Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
>
> > > 1. Make it optional to track chunk memory space by a template parameter. It might be tiny compared to everything else that vacuum does. That would allow other users to avoid that overhead.
> > > 2. When context block usage exceeds the limit (rare), make the additional effort to get the precise usage -- I'm not sure such a top-down facility exists, and I'm not feeling well enough today to study this further.
> >
> > I prefer option (1) as it's straight forward. I mentioned a similar
> > idea before[1]. RT_MEMORY_USAGE() is defined only when the macro is
> > defined. It might be worth checking if there is visible overhead of
> > tracking chunk memory space. IIRC we've not evaluated it yet.
>
> Ok, let's try this -- I can test and profile later this week.
Thanks!
I've attached the new version patches. I merged improvements and fixes
I did in the v29 patch. 0007 through 0010 are updates from v29. The
main change made in v30 is to make the memory measurement and
RT_MEMORY_USAGE() optional, which is done in 0007 patch. The 0008 and
0009 patches are the updates for tidstore and the vacuum integration
patches. Here are results of quick tests (an average of 3 executions):
query: select * from bench_load_random_int(10 * 1000 * 1000)
* w/ RT_MEASURE_MEMORY_USAGE:
mem_allocated | load_ms
---------------+---------
1996512000 | 3305
(1 row)
* w/o RT_MEASURE_MEMORY_USAGE:
mem_allocated | load_ms
---------------+---------
0 | 3258
(1 row)
It seems to be within a noise level but I agree to make it optional.
Apart from the memory measurement stuff, I've done another todo item
on my list; adding min max classes for node3 and node125. I've done
that in 0010 patch, and here is a quick test result:
query: select * from bench_load_random_int(10 * 1000 * 1000)
* w/ 0000 patch
mem_allocated | load_ms
---------------+---------
1268630080 | 3275
(1 row)
* w/o 0000 patch
mem_allocated | load_ms
---------------+---------
1996512000 | 3214
(1 row)
That's a good improvement on the memory usage, without a noticeable
performance overhead. FYI CLASS_3_MIN has 1 fanout and is 24 bytes in
size, and CLASS_125_MIN has 61 fanouts and is 768 bytes in size.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
From | Date | Subject | |
---|---|---|---|
Next Message | Michael Paquier | 2023-03-09 07:09:49 | Re: Raising the SCRAM iteration count |
Previous Message | Michael Paquier | 2023-03-09 06:46:53 | Re: Testing autovacuum wraparound (including failsafe) |