Re: snapbuild woes

On 12/05/17 03:31, Andres Freund wrote:
> On 2017-05-11 14:54:26 -0700, Andres Freund wrote:
>> On 2017-05-11 14:51:55 -0700, wrote:
>>> On 2017-05-08 00:10:12 -0700, Andres Freund wrote:
>>>> I plan to commit the next pending patch after the back branch releases
>>>> are cut - it's an invasive fix and the issue doesn't cause corruption
>>>> "just" slow slot creation. So it seems better to wait for a few days,
>>>> rather than hurry it into the release.
>>>
>>> Now that that's done, here's an updated version of that patch. Note the
>>> new logic to trigger xl_running_xact's to be logged at the right spot.
>>> Works well in my testing.
>>>
>>> I plan to commit this fairly soon, unless somebody wants a bit more time
>>> to look into it.
>>>
>>> Unless somebody protests, I'd like to slightly revise how the on-disk
>>> snapshots are stored on master - given the issues this bug/commit showed
>>> with the current method - but I can see one could argue that that should
>>> rather be done next release.
>>
>> As usual I forgot my attachement.
>
> This patch also seems to offer a way to do your 0005 in, possibly, more
> efficient manner. We don't ever need to assume a transaction is
> timetravelling anymore. Could you check whether the attached, hasty,
> patch resolves the performance problems you measured? Also, do you have
> a "reference" workload for that?
>

Hmm, well it helps but actually now that we don't track individual
running transactions anymore it got much less effective (my version of
0005 does as well).

The example workload I test with is:
session 1: open transaction, do a write, keep it open
session 2: pgbench -M simple -N -c 10 -P 1 -T 5
session 3: run CREATE_REPLICATION_SLOT LOGICAL in walsender
session 2: pgbench -M simple -N -c 10 -P 1 -T 20
session 1: commit

And wait for session 3 to finish slot creation, takes about 20 mins on
my laptop without patches, minute and half with your patches for 0004
and 0005 (or with your 0004 and my 0005) and about 2s with my original
0004 and 0005.

What makes it slow is the constant qsorting IIRC.

--
Petr Jelinek http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services