Re: ERROR: invalid memory alloc request size 1073741824

On 01/30/2018 06:35 PM, Stefan Blanke wrote:
> Hello,
>
> We've tripped over an error when doing a "COPY.. TO STDOUT WITH BINARY"
> query.
>
> "ERROR:  invalid memory alloc request size 1073741824"
> (exactly 1GB)
>
> So a palloc() call is failing on the AllocSizeIsValid() check.
>
> Does anyone know if this a safety catch we are tripping with a bad
> query - or whether this check is something that should never be hit
> (i.e. a bug).
>

That very much depends on where exactly the failure happens - it might
be both (or relying on an assumption that changed over). It's not clear
if it fails during query execution or formatting the output (I seem to
remember there were/are issues with rows containing multiple large bytea
values, for example).

Can you attach gdb to the backend, set breakpoint on that elog and share
the backtrace? Then we can investigate further.

In case you don't have experience with doing that, here's a howto on the
community wiki:

https://wiki.postgresql.org/wiki/Getting_a_stack_trace_of_a_running_PostgreSQL_backend_on_Linux/BSD

> This has been some discussion before about making a change before the
> 9.5beta1:
>
> https://www.postgresql.org/message-id/flat/9A28C8860F777E439AA12E8AEA7694F8010F6F3F%40BPXM15GP.gisp.nec.co.jp
>
> https://www.postgresql.org/message-id/flat/CAKJS1f_sxPyBHF6%3DnEYwPJdKKDNjVDP91b_EJApfyGTei%3DViDw%40mail.gmail.com
>

Both of those threads are about a hashjoin issue, which should have been
fixed in 9.5. Moreover, you haven't shared the query and it's query
plan, so it's unclear if it's doing hash joins at all. That would be
useful for investigating this issue, BTW.

>
> postgresql 9.5.4 (2PGDG.rhel6)
> work_mem = 256MB
>

FWIW you're missing more than a year of bugfixes

regards

--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services