Re: partial heap only tuples

On 3/9/21, 8:24 AM, "Bruce Momjian" <bruce(at)momjian(dot)us> wrote:
> On Mon, Feb 15, 2021 at 08:19:40PM +0000, Bossart, Nathan wrote:
>> Yeah, this is something I'm concerned about. I think adding a bitmap
>> of modified columns to the header of PHOT-updated tuples improves
>> matters quite a bit, even for single-page vacuuming. Following is a
>> strategy I've been developing (there may still be some gaps). Here's
>> a basic PHOT chain where all tuples are visible and the last one has
>> not been deleted or updated:
>>
>> idx1 0 1 2 3
>> idx2 0 1 2
>> idx3 0 2 3
>> lp 1 2 3 4 5
>> tuple (0,0,0) (0,1,1) (2,2,1) (2,2,2) (3,2,3)
>> bitmap -xx xx- --x x-x
>
> First, I want to continue encouraging you to work on this because I
> think it can yield big improvements. Second, I like the wiki you
> created. Third, the diagram above seems to be more meaningful if read
> from the bottom-up. I suggest you reorder it on the wiki so it can be
> read top-down, maybe:
>
>> lp 1 2 3 4 5
>> tuple (0,0,0) (0,1,1) (2,2,1) (2,2,2) (3,2,3)
>> bitmap -xx xx- --x x-x
>> idx1 0 1 2 3
>> idx2 0 1 2
>> idx3 0 2 3

I appreciate the feedback and the words of encouragement. I'll go
ahead and flip the diagrams like you suggested. I'm planning on
publishing a larger round of edits to the wiki once the patch set is
ready to share. There are a few changes to the design that I've
picked up along the way.

> Fourth, I know in the wiki you said create/drop index needs more
> research, but I suggest you avoid any design that will be overly complex
> for create/drop index. For example, a per-row bitmap that is based on
> what indexes exist at time of row creation might cause unacceptable
> problems in handling create/drop index. Would you number indexes? I am
> not saying you have to solve all the problems now, but you have to keep
> your eye on obstacles that might block your progress later.

I am agreed on avoiding an overly complex design. This project
introduces a certain amount of inherent complexity, so one of my main
goals is ensuring that it's easy to reason about each piece.

I'm cautiously optimistic that index creation and deletion will not
require too much extra work. For example, if a new index needs to
point to a partial heap only tuple, it can do so (unlike HOT, which
would require that the new index point to the root of the chain). The
modified-columns bitmaps could include the entire set of modified
columns (not just the indexed ones), so no additional changes would
need to be made there. Furthermore, I'm anticipating that the
modified-columns bitmaps will end up only being used with the
redirected LPs to help reduce heap bloat after single-page vacuuming.
In that case, new indexes would probably avoid the existing bitmaps
anyway.

Nathan