From: | Chris Mair <chris(at)1006(dot)org> |
---|---|
To: | Marc-André Goderre <magoderre(at)cgq(dot)qc(dot)ca>, "'pgsql-general(at)postgresql(dot)org' (pgsql-general(at)postgresql(dot)org)" <pgsql-general(at)postgresql(dot)org> |
Subject: | Re: Invalid memory alloc |
Date: | 2015-04-23 20:17:51 |
Message-ID: | d51153a2ad075782d0134d5c843c8164@smtp.hushmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-general |
> Hello, I'm processing a 100Million row table.
> I get error message about memory and I'ld like to know what can cause this issue.
>
> ...
> psql:/home/ubuntu/create_topo.sql:12: NOTICE: 104855000 edges processed
> psql:/home/ubuntu/create_topo.sql:12: NOTICE: 104856000 edges processed
> psql:/home/ubuntu/create_topo.sql:12: NOTICE: 104857000 edges processed
> psql:/home/ubuntu/create_topo.sql:12: NOTICE: invalid memory alloc request size 1677721600
> psql:/home/ubuntu/create_topo.sql:12: NOTICE: UPDATE public.way_noded SET source = 88374866,target = 88362922 WHERE id = 142645362
> pgr_createtopology
> --------------------
> FAIL
> (1 row)
>
> The server has a 10Gb of shared_buffer.
> Do you thing this quantity of memory allowed should normaly be enough to process the data?
>
> Thanks
> Marc
Hi,
what version of Postgres are you using? Any extensions? I guess you're using PostGIS, right?
This error indicates something is trying to allocate 1600 MB of memory in the backend - that
should never happen, as data chunks that are larger than 1 GB are broken down in smaller pieces.
I hope you're not suffering data corruption, what happens if you do select * from public.way_noded
where id = 142645362 ?
Any other hints? Log messages (from Linux or the postgres backend)?
Bye,
Chris.
From | Date | Subject | |
---|---|---|---|
Next Message | Jim Nasby | 2015-04-23 20:27:38 | Re: Moving Specific Data Across Schemas Including FKs |
Previous Message | Edson Richter | 2015-04-23 20:15:46 | Re: Invalid memory alloc |