Re: pgsql: Permit super-MaxAllocSize allocations with MemoryContextAllocHug

On Tue, Jul 02, 2013 at 12:08:42PM +0200, Andres Freund wrote:
> On 2013-06-27 19:05:22 +0000, Noah Misch wrote:
> > Permit super-MaxAllocSize allocations with MemoryContextAllocHuge().
> >
> > The MaxAllocSize guard is convenient for most callers, because it
> > reduces the need for careful attention to overflow, data type selection,
> > and the SET_VARSIZE() limit. A handful of callers are happy to navigate
> > those hazards in exchange for the ability to allocate a larger chunk.
> > Introduce MemoryContextAllocHuge() and repalloc_huge(). Use this in
> > tuplesort.c and tuplestore.c, enabling internal sorts of up to INT_MAX
> > tuples, a factor-of-48 increase. In particular, B-tree index builds can
> > now benefit from much-larger maintenance_work_mem settings.
>
> This commit causes a bunch of warnings like:
>
> src/backend/utils/sort/tuplesort.c: In function ???tuplesort_begin_common???:
> src/backend/utils/sort/tuplesort.c:399:33: warning: comparison of unsigned expression < 0 is always false
> [-Wtype-limits]
> #define LACKMEM(state) ((state)->availMem < 0)
>
> to be thrown during compilation. And I think it is spot on. Unless you
> overhaul a good bit of the respective logic making availMem unsigned
> isn't going to fly.

True. Will look into it; thanks.

--
Noah Misch
EnterpriseDB http://www.enterprisedb.com