performance problems with bulk inserts/updates on tsrange with gist-based exclude constrains

From: Chris Withers <chris(at)withers(dot)org>
To: pgsql-general <pgsql-general(at)postgresql(dot)org>
Subject: performance problems with bulk inserts/updates on tsrange with gist-based exclude constrains
Date: 2016-09-16 08:34:43
Message-ID: a85d26e9-166d-d233-e63f-818362825a53@withers.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Hi All,

I have quite a few tables that follow a pattern like this:

Table "public.my_model"
Column | Type | Modifiers
--------+-------------------+-----------
period | tsrange | not null
key | character varying | not null
value | integer |
Indexes:
"my_model_pkey" PRIMARY KEY, btree (period, key)
"my_model_period_key_excl" EXCLUDE USING gist (period WITH &&, key
WITH =)
Check constraints:
"my_model_period_check" CHECK (period <> 'empty'::tsrange)

So, a primary key of a period column and one or more other columns
(usually int or string) and an exclude constraint to prevent overlaps,
and a check constraint to prevent empty ranges.

However, I'm hitting performance problems on moderate bulk inserts and
updates, with ~700k rows taking around 13 minutes. Profiling my python
code suggests that most of the time is being taken by Postgres (9.4 in
this case...)

What can I do to speed things up? Is there a different type of index I
can use to achieve the same exclude constraint? Is there something I can
do to have the index changes only done on the commit of the bulk batches?

cheers,

Chris

Browse pgsql-general by date

  From Date Subject
Next Message Thomas.Deboben.ext 2016-09-16 08:46:16 PostgreSQL installation failure
Previous Message Arun Rangarajan 2016-09-15 23:20:19 Re: Unable to create oracle_fdw (foreign data wrapper) extension