Re: Multiple sorts in a query

On Tue, May 19, 2009 at 7:44 AM, Martijn van Oosterhout
<kleptog(at)svana(dot)org> wrote:
> On Tue, May 19, 2009 at 12:32:13PM +0100, Simon Riggs wrote:
>> If we allocate large chunks of memory we use malloc(). So complex
>> queries can have multiple mallocs, followed by multiple reallocs. That
>> in itself seems likely to end up with roughly double memory use, since
>> realloc won't work properly/quickly with multiple mallocs. (Double since
>> we allocate X bytes, then 2X bytes etc until we hit the limit.)
>
> I don't know about Solaris, but glibc has a threshold above which it
> starts using mmap() instead of sbrk(). Thus, once you start using very
> large blocks, freeing always returns the memory to the kernel,
> irrespective of other allocations.
>
> The threshold is dynamic apparently, but starts at 128KB.

I just read an article that suggests assuming that can be dangerous
(by one of the authors of jemalloc)...an interesting read.

"Update in 2006:
The above was written in 2001. Since then the world has changed a lot.
Memory got bigger. Applications got bigger. The virtual address space
layout in 32 bit linux changed.

In the new situation, brk() and mmap space is shared and there are no
artificial limits on brk size imposed by the kernel. What is more,
applications have started using transient allocations larger than the
128Kb as was imagined in 2001."

http://www.canonware.com/~ttt/2009/05/mr-malloc-gets-schooled.html

merlin