From: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
---|---|
To: | Peter Eisentraut <peter(dot)eisentraut(at)2ndquadrant(dot)com> |
Cc: | Stephen Frost <sfrost(at)snowman(dot)net>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: pg_dump test instability |
Date: | 2018-09-13 21:03:39 |
Message-ID: | 6727.1536872619@sss.pgh.pa.us |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
I wrote:
> Peter Eisentraut <peter(dot)eisentraut(at)2ndquadrant(dot)com> writes:
>> I see. Why not shift all items up to the i'th up by one place, instead
>> of moving only the first one? That way the sortedness would be
>> preserved. Otherwise we'd move the first one into the middle, then
>> sorting would move it to the front again, etc.
> Hmmm ... might be worth doing, but I'm not sure. The steady-state cycle
> will probably be that after one task has been dispatched, we'll sleep
> until some task finishes and then that will unblock some pending items,
> resulting in new entries at the end of the list, forcing a sort anyway
> before we next dispatch a task. So I was expecting that avoiding a sort
> here wasn't really going to be worth expending much effort for. But my
> intuition about that could be wrong. I'll run a test case with some
> instrumentation added and see how often we could avoid sorts by
> memmove'ing.
OK, my intuition was faulty. At least when testing with the regression
database, situations where we are taking the slow path at all seem to
involve several interrelated dump objects (eg indexes of a table) that
are all waiting for the same lock, such that we may be able to dispatch a
number of unrelated tasks before anything gets added from the pending
list. Doing it as you suggest eliminates a significant fraction of
the re-sort operations.
Attached updated patch does it like that and makes the cosmetic
adjustments you suggested. I also went ahead and did the renaming
of par_prev/par_next/par_list_xxx that I'd suggested upthread.
I think this is committable ...
regards, tom lane
Attachment | Content-Type | Size |
---|---|---|
smarter-parallel-dump-restore-2.patch | text/x-diff | 51.7 KB |
From | Date | Subject | |
---|---|---|---|
Next Message | Andrew Gierth | 2018-09-13 21:12:03 | Re: Consistent segfault in complex query |
Previous Message | Jerry Jelinek | 2018-09-13 20:56:42 | Re: patch to allow disable of WAL recycling |