Re: Crash: invalid DSA memory alloc request

Hi Matthias,

On Thu, Dec 12, 2024 at 10:49 PM Matthias van de Meent <
boekewurm+postgres(at)gmail(dot)com> wrote:

> On Thu, 12 Dec 2024 at 22:28, Andreas 'ads' Scherbaum <ads(at)pgug(dot)de> wrote:
> >
> >
> > Hello,
> >
> > I'm running a couple of large tests, and in this particular test I have
> > a few million tables more.
> >
> > At some point it fails, and I gathered the following trace:
> >
> >
> > 2024-12-12 22:22:55.307 CET [1496210] ERROR: invalid DSA memory alloc
> > request size 1073741824
> > 2024-12-12 22:22:55.307 CET [1496210] BACKTRACE:
> > postgres: ads tabletest [local] CREATE TABLE(+0x15e570)
> > [0x6309c379c570]
> > postgres: ads tabletest [local] CREATE
> > TABLE(dshash_find_or_insert+0x1a4) [0x6309c39882d4]
> > postgres: ads tabletest [local] CREATE
> > TABLE(pgstat_get_entry_ref+0x440) [0x6309c3b0a530]
>
> It looks like the dshash table used in the pgstats system uses
> resize(), which only specifies DSA_ALLOC_ZERO, not DSA_ALLOC_HUGE,
> causing issues when the table grows larger than 1 GB.
>
> I expect that error to disappear when you replace the
> dsa_allocate0(...) call in dshash.c's resize function with
> dsa_allocate_extended(..., DSA_ALLOC_HUGE | DSA_ALLOC_ZERO) as
> attached, but haven't tested it due to a lack of database with
> millions of relations.
>

Can confirm that the crash no longer happens when applying your patch.

Was able to both continue the old and crashed test, as well as run a new
test:

tabletest=# select count(*) from information_schema.tables;
count
----------
20000211
(1 row)

Thanks,

--
Andreas 'ads' Scherbaum
German PostgreSQL User Group
European PostgreSQL User Group - Board of Directors
Volunteer Regional Contact, Germany - PostgreSQL Project