Re: Query Failed, out of memory

On Oct 5, 2006, at 11:15 AM, Mark Woodward wrote:
>> On Thu, Oct 05, 2006 at 11:56:43AM -0400, Mark Woodward wrote:
>>> The query was executed as:
>>> psql -p 5435 -U pgsql -t -A -c "select client, item, rating, day
>>> from
>>> ratings order by client" netflix > netflix.txt
>>>
>>>
>>> My question, it looks like the kernel killed psql, and not
>>> postmaster.
>>> The
>>> postgresql log file complained about a broken pipe.
>>>
>>> Question, is this a bug in psql? It took over 4 hours of run time
>>> before
>>> the crash.
>>
>> Well, psql tried to store the entire resultset in memory at once, and
>> failed. I'm not sure how many records you were trying to display, but
>> try to estimate how much memory that would take to store...
>>
>> What were you trying to do?
>>
> It's the stupid NetFlix prize thing, I need to dump out the data in a
> specific order. This is just *one* such query I want to try. I
> guess, like
> I told Tom, I have to write a small program that uses a cursor. :-(

IIRC, 8.2 adds the ability to at least copy from a view, if not a raw
SELECT, so you should probably do that instead. Plus it'd be good to
bang on 8.2 with that data set. :) You'll also likely get better
performance.
--
Jim Nasby jim(at)nasby(dot)net
EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)