IndexJoin memory problem using spgist and boxes

From: Anton Dignös <dignoes(at)inf(dot)unibz(dot)it>
To: pgsql-hackers(at)lists(dot)postgresql(dot)org
Subject: IndexJoin memory problem using spgist and boxes
Date: 2018-02-01 08:43:14
Message-ID: CALNdv1jb6y2Te-m8xHLxLX12RsBmZJ1f4hESX7J0HjgyOhA9eA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

I came across a strange memory problem when doing an IndexJoin using
spgist on boxes.
I also found it mentioned here:
https://www.postgresql.org/message-id/flat/CAPqRbE5vTGWCGrOc91Bmu-0o7CwsU0UCnAshOtpDR8cSpSjy0g%40mail(dot)gmail(dot)com#CAPqRbE5vTGWCGrOc91Bmu-0o7CwsU0UCnAshOtpDR8cSpSjy0g(at)mail(dot)gmail(dot)com

With the following setting:

CREATE TABLE r AS SELECT 1 i, box(point(generate_series,
generate_series), point(generate_series+10, generate_series+10)) FROM
generate_series(1, 1000000);
CREATE TABLE s AS SELECT 1 i, box(point(generate_series,
generate_series), point(generate_series+10, generate_series+10)) FROM
generate_series(1, 1000000) ORDER BY random(); -- random sort just
speeds up index creation
CREATE INDEX s_idx ON s USING spgist(box);

postgres consumes several GBs of main memory for the following query:

SELECT * FROM r, s WHERE r.box && s.box;

The problem also occurs for polygons which are based on boxes and are
now part of the dev version.

The attached patch should fix this problem by maintaining the
traversal memory per index scan instead of for the entire join.

Best regards,
Anton

Attachment Content-Type Size
spgist-mem-fix.diff text/plain 1.1 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tomas Vondra 2018-02-01 08:45:50 Re: STATISTICS retained in CREATE TABLE ... LIKE (INCLUDING ALL)?
Previous Message Michael Paquier 2018-02-01 08:20:33 Re: [HACKERS] Creating backup history files for backups taken from standbys