Re: BUG #14051: GIN index creation fails with large number of duplicate keys

On Tue, Mar 29, 2016 at 4:50 AM, <daveg(at)sonic(dot)net> wrote:
> The following bug has been logged on the website:
>
> Bug reference: 14051
> Logged by: David Gould
> Email address: daveg(at)sonic(dot)net
> PostgreSQL version: 9.4.6
> Operating system: Linux
> Description:
>
> Creating a GIN index that contains a large number of duplicate keys fails.
>
> create table ginfail as
> select i::int, array[1::int] as manydups
> from generate_series(1, 180000000) x(i);
> SELECT 180000000
>
> create index ginfail_gin on ginfail using gin(manydups);
> ERROR: invalid memory alloc request size 2013265920
>
...

>
> The problem appears to be that the repalloc() of eo->list to double the size
> of the allocation runs over MaxAllocSize. Perhaps it could use
> repalloc_huge() instead?

In 9.6, that is what it does (commit
30bb26b5e04cff911db960801e32e5f57045eb61). But I don't think there
are any plans to back-port it to 9.4 and 9.5.

Although it looks like it should be pretty easy to do that back-patching.

Cheers,

Jeff