Re: [WIP] speeding up GIN build with parallel workers

From: Dmitry Ivanov <d(dot)ivanov(at)postgrespro(dot)ru>
To: pgsql-hackers(at)postgresql(dot)org
Cc: "Constantin S(dot) Pan" <kvapen(at)gmail(dot)com>
Subject: Re: [WIP] speeding up GIN build with parallel workers
Date: 2016-03-18 17:40:16
Message-ID: 2503839.W1o5kfb8lG@abook
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi Constantin,

I did a quick review of your patch, and here are my comments:

- This patch applies cleanly to the current HEAD (61d2ebdbf91).

- Code compiles without warnings.

- Currently there's no documentation regarding parallel gin build feature and
provided GUC variables.

- Built indexes work seem to work normally.

Performance
-----------

I've made a few runs on my laptop (i7-4710HQ, default postgresql.conf), here
are the results:

workers avg. time (s)
0 412
4 133
8 81

Looks like 8 workers & a backend do the job 5x times faster than a sole
backend, which is good!

Code style
----------

There are some things that you've probably overlooked, such as:

> task->heap_oid = heap->rd_id;
> task->index_oid = index->rd_id;

You could replace direct access to 'rd_id' field with the RelationGetRelid
macro.

> static void ginDumpEntry(GinState *ginstate,
> volatile WorkerResult *r

Parameter 'r' is unused, you could remove it.

Some of the functions and pieces of code that you've added do not comply to
the formatting conventions, e. g.

> static int claimSomeBlocks(...
> static GinEntryStack *pushEntry(

>> // The message consists of 2 or 3 parts. iovec allows us to send them as

etc.

Keep up the good work!

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Andres Freund 2016-03-18 17:53:36 Re: Performance degradation in commit ac1d794
Previous Message Teodor Sigaev 2016-03-18 17:24:35 Re: [PATCH] we have added support for box type in SP-GiST index