Re: ERROR: invalid memory alloc request size 1073741824

On Wed, Jan 31, 2018 at 12:32 PM, Stefan Blanke <
stefan(dot)blanke(at)framestore(dot)com> wrote:

> >
> > I'll bet you it's not that. It's quite unlikely that would fail with
> > exactly 1GB request size. It seems much more like a buffer that we keep
> > to be power of 2. The question is which one.
>
> I had dismissed corruption before writing in. It's exactly 1GB every time
> this has happened - and we can dump the full dataset periodically without
> issue.
>
> >> I have my money on a corrupted TOAST entry. Is this happening on
> >> trustworthy hardware or beige box with no ECC or RAID?
>
> It's good quality commercial hardware in our colo - no exactly sure what.
>

If it is a sporadic issue and you can dump the full dataset, then I just
lost my money (Tomas, you coming to PGConf in Jersey City?).

But then, if this is a plain COPY to stdout ... I am wondering what could
possibly be in that path that wants to allocate 1GB. Or is this not so
plain but rather a COPY ... SELECT ...?

Regards, Jan

>
>
> Thanks for taking the time to look at this!
> Stefan
>
>
>
>
>
> On 01/30/18 22:00, Tomas Vondra wrote:
>
>>
>>
>> On 01/30/2018 10:43 PM, Jan Wieck wrote:
>>
>>>
>>>
>>> On Tue, Jan 30, 2018 at 12:35 PM, Stefan Blanke
>>> <stefan(dot)blanke(at)framestore(dot)com <mailto:stefan(dot)blanke(at)framestore(dot)com>>
>>> wrote:
>>>
>>> Hello,
>>>
>>> We've tripped over an error when doing a "COPY.. TO STDOUT WITH
>>> BINARY" query.
>>>
>>> "ERROR: invalid memory alloc request size 1073741824"
>>> (exactly 1GB)
>>>
>>>
>>> I have my money on a corrupted TOAST entry. Is this happening on
>>> trustworthy hardware or beige box with no ECC or RAID?
>>>
>>>
>> I'll bet you it's not that. It's quite unlikely that would fail with
>> exactly 1GB request size. It seems much more like a buffer that we keep
>> to be power of 2. The question is which one.
>>
>>
>> regards
>>
>>

--
Jan Wieck
Senior Postgres Architect
http://pgblog.wi3ck.info