Re: Automatically setting work_mem

On Tue, Mar 21, 2006 at 08:05:50PM +0000, Simon Riggs wrote:
> > Point 2 is actually a serious flaw in Simon's proposal, because there
> > is no portable way to make malloc return freed memory to the OS. Some
> > mallocs will do that ... in some cases ... but many simply don't ever
> > move the brk address down. It's not an easy thing to do when the arena
> > gets cluttered with a lot of different alloc chunks and only some of
> > them get freed.

<snip>

> The largest requirement for memory is the run building during
> performsort. That portion of the code is not concurrently executed
> within the same query. If we can reduce memory usage after that phase
> completes then we stand a chance of not overusing memory on a big query
> and not being able to reclaim it.

There is one way to guarentee the memory is released to the OS after
completion. Make the allocator allocate work_mem bytes using mmap()
rather than malloc(). munmap() will then definitly return the memory to
the OS. Unfortunatly, the coding required would probably not be
straight-forward... Glibc will only convert malloc() to an mmap() on
allocations > 128KB and I don't think PostgreSQL ever does that.

Have a ncie day,
--
Martijn van Oosterhout <kleptog(at)svana(dot)org> http://svana.org/kleptog/
> Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a
> tool for doing 5% of the work and then sitting around waiting for someone
> else to do the other 95% so you can sue them.