Re: Slow GIN indexes after bulk insert

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Chris Spencer <chrisspen(at)gmail(dot)com>
Cc: pgsql-general(at)postgresql(dot)org
Subject: Re: Slow GIN indexes after bulk insert
Date: 2016-03-21 19:36:37
Message-ID: 29866.1458588997@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Chris Spencer <chrisspen(at)gmail(dot)com> writes:
> What constitutes a "large" work_mem? My server has 61GB of memory and my
> work_mem is currently set to include all of that.

Ouch. That's a mistake independently of GIN. The primary usage of
work_mem is to define how much memory an individual sorting or hashing
query step is allowed to use. A complex query might have several sort or
hash steps, and then you need to worry about concurrent queries in
different sessions; not to mention that this is not the only demand on
your server's RAM. I'd be hesitant to set work_mem much above 1GB, maybe
even quite a bit less than that depending on what your workload is like.

Cutting work_mem to ~100MB might alone be enough to fix your GIN issue;
if not you could experiment with forced flushes of the GIN pending lists
via VACUUM (or ANALYZE might do it too, and be more directly useful).

regards, tom lane

In response to

Browse pgsql-general by date

  From Date Subject
Next Message Tom Lane 2016-03-21 21:25:04 Re: [GENERAL] Request - repeat value of \pset title during \watch interations
Previous Message Chris Spencer 2016-03-21 19:28:37 Re: Slow GIN indexes after bulk insert