pgsql: Improve memory management for external sorts.

From: Robert Haas <rhaas(at)postgresql(dot)org>
To: pgsql-committers(at)postgresql(dot)org
Subject: pgsql: Improve memory management for external sorts.
Date: 2016-03-17 20:11:00
Message-ID: E1ageG4-0001Sj-RW@gemulon.postgresql.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-committers pgsql-hackers

Improve memory management for external sorts.

Introduce a new memory context which stores tuple data, and reset it
at the end of each merge pass; this helps avoid memory fragmentation
and, consequently, overallocation. Also, for the final merge patch,
eliminate memory context chunk header overhead entirely by allocating
all of the memory used for buffering tuples during the merge in a
single chunk. Since this modestly increases the number of tuples we
can store, grow the memtuples array a bit so that we're less likely to
run short of slots there.

Peter Geoghegan. Review and testing of patches in this series by
Jeff Janes, Greg Stark, Mithun Cy, and me.

Branch
------
master

Details
-------
http://git.postgresql.org/pg/commitdiff/0011c0091e886b874e485a46ff2c94222ffbf550

Modified Files
--------------
src/backend/utils/sort/tuplesort.c | 556 ++++++++++++++++++++++++++++++++++---
1 file changed, 516 insertions(+), 40 deletions(-)

Responses

Browse pgsql-committers by date

  From Date Subject
Next Message Robert Haas 2016-03-17 20:12:42 pgsql: Update tuplesort.c comments for memory mangement improvements.
Previous Message Tom Lane 2016-03-17 19:50:51 pgsql: Fix assorted breakage in to_char()'s OF format option.

Browse pgsql-hackers by date

  From Date Subject
Next Message Robert Haas 2016-03-17 20:13:26 Re: Using quicksort for every external sort run
Previous Message Pavel Stehule 2016-03-17 20:02:59 Re: IF (NOT) EXISTS in psql-completion