From: | Marc-André Goderre <magoderre(at)cgq(dot)qc(dot)ca> |
---|---|
To: | Edson Richter <edsonrichter(at)hotmail(dot)com>, "pgsql-general(at)postgresql(dot)org" <pgsql-general(at)postgresql(dot)org> |
Subject: | Re: Invalid memory alloc |
Date: | 2015-04-24 17:49:06 |
Message-ID: | D8DAC91BA6256246BA3FF835F6AAA1189943FB99@SI-MB01.cegep-chicoutimi.qc.ca |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-general |
Server setting
kernel.shmmax = 13322600448
kernel.shmall = 4194304
vm.swappiness = 60
vm.overcommit_memory = 0
vm.overcommit_ratio = 50
Postgresql setting
effective_cache_size | 128MB
checkpoint_segments | 64
maintenance_work_mem | 5GB
segment_size | 1GB
shared_buffers | 10GB
work_mem | 256MB
I use Postgis and PGrouting extension.
The error come when I use a pgrouting function pgr_createtopology()
Marc
Marc-André Goderre
Analyste en informatique
Courriel: magoderre(at)cgq(dot)qc(dot)ca
Tel.: (418) 698-5995 poste 1628
Téléc.: (418) 698-4108
Cégep de Chicoutimi, 534 rue Jacques-Cartier Est, Chicoutimi (Québec) G7H 1Z6
Visitez notre site web au : www.cgq.qc.ca
Suivez-nous sur Twitter : @lecgq
AVIS DE CONFIDENTIALITÉ :
Le présent courriel peut contenir des renseignements confidentiels s’adressant qu’au destinataire ou à celui ont le nom figure ci-dessus. Si ce courriel vous est parvenu par mégarde, veuillez le supprimer et nous en aviser aussitôt par téléphone. Merci.
Pensez à l'environnement avant d'imprimer ce message
-----Message d'origine-----
De : pgsql-general-owner(at)postgresql(dot)org [mailto:pgsql-general-owner(at)postgresql(dot)org] De la part de Edson Richter
Envoyé : 23 avril 2015 16:16
À : pgsql-general(at)postgresql(dot)org
Objet : Re: [GENERAL] Invalid memory alloc
On 23-04-2015 16:55, Marc-André Goderre wrote:
> Hello, I'm processing a 100Million row table.
> I get error message about memory and I'ld like to know what can cause this issue.
>
> ...
> psql:/home/ubuntu/create_topo.sql:12: NOTICE: 104855000 edges
> processed
> psql:/home/ubuntu/create_topo.sql:12: NOTICE: 104856000 edges
> processed
> psql:/home/ubuntu/create_topo.sql:12: NOTICE: 104857000 edges
> processed
> psql:/home/ubuntu/create_topo.sql:12: NOTICE: invalid memory alloc
> request size 1677721600
> psql:/home/ubuntu/create_topo.sql:12: NOTICE: UPDATE public.way_noded SET source = 88374866,target = 88362922 WHERE id = 142645362
> pgr_createtopology
> --------------------
> FAIL
> (1 row)
>
> The server has a 10Gb of shared_buffer.
> Do you thing this quantity of memory allowed should normaly be enough to process the data?
>
> Thanks
> Marc
>
>
My question would sound stupid... you have 10Gb shared buffer, but how much physical memory on this server?
How have you configured the kernel swappines, overcommit_memoryt, overcommit_ratio?
Have you set anything different in shmmax or shmall?
Edson
--
Sent via pgsql-general mailing list (pgsql-general(at)postgresql(dot)org) To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general
From | Date | Subject | |
---|---|---|---|
Next Message | Christoph Berg | 2015-04-24 17:57:04 | Re: Running pg_upgrade under Debian |
Previous Message | Marc-André Goderre | 2015-04-24 14:15:05 | Re: Invalid memory alloc |