Re: strange slow query - lost lot of time somewhere

On Thu, 5 May 2022 at 19:26, Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com> wrote:
>
> čt 5. 5. 2022 v 8:51 odesílatel Jakub Wartak <Jakub(dot)Wartak(at)tomtom(dot)com> napsal:
>> > Breakpoint 1, 0x00007f557f0c16c0 in mmap64 () from /lib64/libc.so.6
>> > (gdb) bt
>> > #0 0x00007f557f0c16c0 in mmap64 () from /lib64/libc.so.6
>> > #1 0x00007f557f04dd91 in sysmalloc () from /lib64/libc.so.6
>> > #2 0x00007f557f04eaa9 in _int_malloc () from /lib64/libc.so.6
>> > #3 0x00007f557f04fb1e in malloc () from /lib64/libc.so.6
>> > #4 0x0000000000932134 in AllocSetAlloc ()
>> > #5 0x00000000009376cf in MemoryContextAllocExtended ()
>> > #6 0x00000000006ad915 in ExecInitMemoize ()
>>
>> Well the PGDG repo have the debuginfos (e.g. postgresql14-debuginfo) rpms / dpkgs(?) so I hope you are basically 1 command away of being able to debug it further what happens in ExecInitMemoize()
>> Those packages seem to be safe as they modify only /usr/lib/debug so should not have any impact on production workload.
>
> I just have to wait for admin action - I have no root rights for the server.

Looking at ExecInitMemoize() it's hard to see what could take such a
long time other than the build_hash_table(). Tom did mention this,
but I can't quite see how the size given to that function could be
larger than 91 in your case.

If you get the debug symbols installed, can you use gdb to

break nodeMemoize.c:268
p size

maybe there's something I'm missing following the code and maybe there
is some way that est_entries is not set to what I thought it was.

It would also be good to see the same perf report again after the
debug symbols are installed in order to resolve those unresolved
function names.

David