Re: Polyphase merge is obsolete

From: Heikki Linnakangas <hlinnaka(at)iki(dot)fi>
To: pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Cc: Peter Geoghegan <pg(at)bowt(dot)ie>, Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>
Subject: Re: Polyphase merge is obsolete
Date: 2021-01-22 22:19:28
Message-ID: 4c6a8fd7-1902-fc8e-e8d6-281d426f5da8@iki.fi
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 22/10/2020 14:48, Heikki Linnakangas wrote:
> On 11/09/2017 13:37, Tomas Vondra wrote:
>> I planned to do some benchmarking on this patch, but apparently the
>> patch no longer applies. Rebase please?
>
> Here's a rebase of this. Sorry to keep you waiting :-).

Here's an updated version that fixes one bug:

The CFBot was reporting a failure on the FreeBSD system [1]. It turned
out to be an out-of-memory issue caused by an underflow bug in the
calculation of the size of the tape read buffer size. With a small
work_mem size, the memory left for tape buffers was negative, and that
wrapped around to a very large number. I believe that was not caught by
the other systems, because the other ones had enough memory for the
incorrectly-sized buffers anyway. That was the case on my laptop at
least. It did cause a big slowdown in the 'tuplesort' regression test
though, which I hadn't noticed.

The fix for that bug is here as a separate patch for easier review, but
I'll squash it before committing.

[1] https://cirrus-ci.com/task/6699842091089920

- Heikki

Attachment Content-Type Size
v3-0001-Refactor-LogicalTapeSet-LogicalTape-interface.patch text/x-patch 67.8 KB
v3-0002-Replace-polyphase-merge-algorithm-with-a-simple-b.patch text/x-patch 40.8 KB
v3-0003-Fix-sizing-of-tape-read-buffers.patch text/x-patch 890 bytes

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2021-01-22 22:33:06 Re: a verbose option for autovacuum
Previous Message Paul Martinez 2021-01-22 22:08:02 Re: Why does creating logical replication subscriptions require superuser?