Query Using Massive Temp Space

From: Cory Tucker <cory(dot)tucker(at)gmail(dot)com>
To: "pgsql-general(at)postgresql(dot)org" <pgsql-general(at)postgresql(dot)org>
Subject: Query Using Massive Temp Space
Date: 2017-11-20 06:16:48
Message-ID: CAG_=8kBoWY4AXwW=Cj44xe13VZnYohV9Yr-_hvZdx2xpiipr9w@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Hello,

I have a query that is using a tremendous amount of temp disk space given
the overall size of the dataset. I'd love for someone to try to explain
what PG is doing and why its using so much space for the query.

First off, the system is PG 9.6 on Ubuntu with 4 cores and 28 GB of RAM.
The query in question is a fairly large join of several tables (6)
including some aggregations. The overall dataset size of the 6 tables in
question is about 20GB and the largest table is about 15M rows. The query
is essentially a dump of *most* of the data from these tables joined
together to be used in another system.

When the query runs it begins to use an aggressive amount of temp space on
the volume over the course of many hours, until it reaches about 95%
capacity and then tapers off. Never completes though. The temp space it
uses is around 1.5TB out of a 2TB volume. Again, the *total* size of the
relations in question is only 20GB.

Can anyone explain how the query could possibly use so much temp space?

Query and query plan are attached. Please let me know if any more info is
needed.

thanks
--Cory

Attachment Content-Type Size
query application/octet-stream 2.7 KB
query_plan application/octet-stream 3.2 KB

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Laurenz Albe 2017-11-20 07:08:12 Re: Query Using Massive Temp Space
Previous Message Tom Lane 2017-11-20 03:48:04 Re: How to store multiple rows in array .