Re: Combine Prune and Freeze records emitted by vacuum

From: Melanie Plageman <melanieplageman(at)gmail(dot)com>
To: Heikki Linnakangas <hlinnaka(at)iki(dot)fi>
Cc: Pg Hackers <pgsql-hackers(at)postgresql(dot)org>, Andres Freund <andres(at)anarazel(dot)de>, Robert Haas <robertmhaas(at)gmail(dot)com>, Peter Geoghegan <pg(at)bowt(dot)ie>
Subject: Re: Combine Prune and Freeze records emitted by vacuum
Date: 2024-03-29 05:04:52
Message-ID: 20240329050452.qx5gd2tcwe2fk4io@liskov
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Thu, Mar 28, 2024 at 11:07:10AM -0400, Melanie Plageman wrote:
> These comments could use another pass. I had added some extra
> (probably redundant) content when I thought I was refactoring it a
> certain way and then changed my mind.
>
> Attached is a diff with some ideas I had for a bit of code simplification.
>
> Are you working on cleaning this patch up or should I pick it up?

Attached v9 is rebased over master. But, more importantly, I took
another pass at heap_prune_chain() and am pretty happy with what I came
up with. See 0021. I simplified the traversal logic and then grouped the
chain processing into three branches at the end. I find it much easier
to understand what we are doing for different types of HOT chains.

I got rid of revisited. We can put it back, but I was thinking: we stash
all HOT tuples and then loop over them later, calling record_unchanged()
on the ones that aren't marked. But, if we have a lot of HOT tuples, is
this really that much better than just looping through all the offsets
and calling record_unchanged() on just the ones that aren't marked?

I've done that in my version. While testing this, I found that only
on-access pruning needed this final loop calling record_unchanged() on
items not yet marked. I know we can't skip this final loop entirely in
the ON ACCESS case because it calls record_prunable(), but we could
consider moving that back out into heap_prune_chain()? Or what do you
think?

I haven't finished updating all the comments, but I am really interested
to know what you think about heap_prune_chain() now.

Note that patches 0001-0020 are still the same as before. Only 0021 is
the new changes I made (they are built on top of your v8 0022).

Tomorrow I will start first thing figuring out how to break this down
into parts that can apply on master and then rebase the rest of the
patches on top of it.

- Melanie

Attachment Content-Type Size
v9-0001-lazy_scan_prune-tests-tuple-vis-with-GlobalVisTes.patch text/x-diff 2.1 KB
v9-0002-Pass-heap_prune_chain-PruneResult-output-paramete.patch text/x-diff 3.3 KB
v9-0003-Rename-PruneState-snapshotConflictHorizon-to-late.patch text/x-diff 2.5 KB
v9-0004-heap_page_prune-sets-all_visible-and-visibility_c.patch text/x-diff 19.0 KB
v9-0005-Add-reference-to-VacuumCutoffs-in-HeapPageFreeze.patch text/x-diff 4.7 KB
v9-0006-Prepare-freeze-tuples-in-heap_page_prune.patch text/x-diff 11.8 KB
v9-0007-lazy_scan_prune-reorder-freeze-execution-logic.patch text/x-diff 5.9 KB
v9-0008-Execute-freezing-in-heap_page_prune.patch text/x-diff 31.4 KB
v9-0009-Make-opp-freeze-heuristic-compatible-with-prune-f.patch text/x-diff 4.4 KB
v9-0010-Separate-tuple-pre-freeze-checks-and-invoke-earli.patch text/x-diff 7.4 KB
v9-0011-Remove-heap_freeze_execute_prepared.patch text/x-diff 8.3 KB
v9-0012-Merge-prune-and-freeze-records.patch text/x-diff 11.6 KB
v9-0013-Set-hastup-in-heap_page_prune.patch text/x-diff 7.7 KB
v9-0014-Count-tuples-for-vacuum-logging-in-heap_page_prun.patch text/x-diff 15.7 KB
v9-0015-Save-dead-tuple-offsets-during-heap_page_prune.patch text/x-diff 6.8 KB
v9-0016-move-live-tuple-accounting-to-heap_prune_chain.patch text/x-diff 41.7 KB
v9-0017-Move-frozen-array-to-PruneState.patch text/x-diff 5.7 KB
v9-0018-Cosmetic-fixes.patch text/x-diff 1.9 KB
v9-0019-Almost-cosmetic-fixes.patch text/x-diff 1.3 KB
v9-0020-Move-frz_conflict_horizon-to-tighter-scope.patch text/x-diff 3.1 KB
v9-0021-WIP-refactor.patch text/x-diff 47.8 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message vignesh C 2024-03-29 05:08:57 Re: Improve eviction algorithm in ReorderBuffer
Previous Message Жарков Роман 2024-03-29 04:49:55 Re: type cache cleanup improvements