Re: [COMMITTERS] pgsql: Improve memory management for external sorts.

From: Robert Haas <robertmhaas(at)gmail(dot)com>
To: Andres Freund <andres(at)anarazel(dot)de>
Cc: "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [COMMITTERS] pgsql: Improve memory management for external sorts.
Date: 2016-03-18 18:40:33
Message-ID: CA+TgmobkDRj0eBa9WG17YcwSmJfF25NLf0X4Y_=iVkX80S5bqw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-committers pgsql-hackers

On Fri, Mar 18, 2016 at 2:39 PM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> On Fri, Mar 18, 2016 at 2:25 PM, Andres Freund <andres(at)anarazel(dot)de> wrote:
>> On 2016-03-17 20:11:00 +0000, Robert Haas wrote:
>>> Improve memory management for external sorts.
>>>
>>> Introduce a new memory context which stores tuple data, and reset it
>>> at the end of each merge pass; this helps avoid memory fragmentation
>>> and, consequently, overallocation. Also, for the final merge patch,
>>> eliminate memory context chunk header overhead entirely by allocating
>>> all of the memory used for buffering tuples during the merge in a
>>> single chunk. Since this modestly increases the number of tuples we
>>> can store, grow the memtuples array a bit so that we're less likely to
>>> run short of slots there.
>>>
>>> Peter Geoghegan. Review and testing of patches in this series by
>>> Jeff Janes, Greg Stark, Mithun Cy, and me.
>>
>> Cross compiling for windows results in:
>> /home/andres/src/postgresql/src/backend/utils/sort/tuplesort.c: In function ‘beginmerge’:
>> /home/andres/src/postgresql/src/backend/utils/sort/tuplesort.c:2695:34: warning: format ‘%ld’ expects argument of type ‘long int’, but argument 4 has type ‘int64 {aka long long int}’ [-Wformat=]
>> elog(LOG, "tape %d initially used %ld KB of %ld KB batch "
>> ^
>> /home/andres/src/postgresql/src/backend/utils/sort/tuplesort.c:2695:34: warning: format ‘%ld’ expects argument of type ‘long int’, but argument 5 has type ‘int64 {aka long long int}’ [-Wformat=]
>> config.status: creating src/interfaces/ecpg/include/ecpg_config.h
>>
>> /home/andres/src/postgresql/src/backend/utils/sort/tuplesort.c: In
>> function ‘beginmerge’:
>> /home/andres/src/postgresql/src/backend/utils/sort/tuplesort.c:2695:34:
>> warning: format ‘%ld’ expects argument of type ‘long int’, but argument
>> 4 has type ‘int64 {aka long long int}’ [-Wformat=]
>> elog(LOG, "tape %d initially used %ld KB of %ld KB batch "
>>
>> Which seems like a valid complain on a LLP64 platform (i.e. where long
>> is 32bit) like windows.
>
> Oops. Thanks for the report. Does this fix it?

Trying again to attach the patch.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Attachment Content-Type Size
int64-fmt-fixup.patch application/x-download 787 bytes

In response to

Responses

Browse pgsql-committers by date

  From Date Subject
Next Message Andres Freund 2016-03-18 18:43:35 Re: [COMMITTERS] pgsql: Improve memory management for external sorts.
Previous Message Robert Haas 2016-03-18 18:39:55 Re: [COMMITTERS] pgsql: Improve memory management for external sorts.

Browse pgsql-hackers by date

  From Date Subject
Next Message Andres Freund 2016-03-18 18:43:35 Re: [COMMITTERS] pgsql: Improve memory management for external sorts.
Previous Message Robert Haas 2016-03-18 18:39:55 Re: [COMMITTERS] pgsql: Improve memory management for external sorts.