Re: out of memory - no sort

On Aug 31, 2011, at 10:52 AM, Don wrote:

> I had always thought that a 32bit machine could access up to 4GB.
> So what is the limiting factor ?

- Half of your memory space may be given over to memory-mapped I/O. Now you're down to 2GB.

- Your process's executable, plus any libraries it uses, plus all the system libraries that they touch, recursively all the way down, are mapped into this space. Now you're likely down to 1.5GB or less free.

- Then of course your process allocates various data structures for each row, even if it's just a huge array of pointers to each row, that would be overhead. And of course the overhead is not nearly that simple--there will be allocations for & pointers to varchars, and info about columns and data types, and heap data structures to keep track of allocated vs free blocks.

- Memory will be fragmented of course, so you can't even use all of what's left.

So no, you can't manipulate 32M of anything except plain numbers or very simple structs in RAM in a 32-bit process.

--
Scott Ribe
scott_ribe(at)elevated-dev(dot)com
http://www.elevated-dev.com/
(303) 722-0567 voice