Re: free(): invalid pointer , when to dumping data to a new server

________________________________
From: Lars Aksel Opsahl <Lars(dot)Opsahl(at)nibio(dot)no>
Sent: Thursday, October 22, 2020 9:32 AM
To: pgsql-admin(at)lists(dot)postgresql(dot)org <pgsql-admin(at)lists(dot)postgresql(dot)org>
Subject: free(): invalid pointer , when to dumping data to a new server

<>Hi
<>
<>
<>I have problem with a new server that crashes when restoring data to it. It's around 1.5 TB of raw data with both spatial and raster data.
<>

I have done some more testing and postgres also crashes right after 'pg_restore --section=pre-data -1' is done also. Here I also see the 'free(): invalid pointer' in the postgres log.

This happens on with Postgres 12 and Postgres 13. and any version of postgis I have tested with.

The pre-data input is little bit more than 200.000 lines with more than 6000 CREATE statements, if I test with the first 100 lines it does not crash. It seems like no data are lost here because the server does recovery at startup.

I tested on another with CentOS and the same software and it worked ok

The server diagnostic does not indicate any memory errors.

Dnf search does not give mee any debuginfo file for postgres

* yum search debuginfo|grep postgres
Last metadata expiration check: 0:17:59 ago on Mon 26 Oct 2020 10:53:03 AM CET.
postgresql12-odbc-debuginfo.x86_64 : Debug information for package postgresql12-odbc

From gdb I get this

gdb /usr/pgsql-12/bin/postmaster /var/lib/systemd/coredump/core.postmaster.26.6f7d98107f544085a41c6f8b876418d0.3832.1603701965000000
GNU gdb (GDB) Red Hat Enterprise Linux 8.2-11.el8
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/pgsql-12/bin/postmaster...Reading symbols from .gnu_debugdata for /usr/pgsql-12/bin/postgres...(no debugging symbols found)...done.
(no debugging symbols found)...done.
BFD: warning: /var/lib/systemd/coredump/core.postmaster.26.6f7d98107f544085a41c6f8b876418d0.3832.1603701965000000 is truncated: expected core file size >= 2195427328, found: 2147483648

warning: core file may not match specified executable file.
[New LWP 3832]
Cannot access memory at address 0x7fdbd7d42128
Cannot access memory at address 0x7fdbd7d42120
Failed to read a valid object file image from memory.
Core was generated by `postgres: postgres sl [local] idle '.
Program terminated with signal SIGABRT, Aborted.
#0 0x00007fdbd508b70f in ?? ()
Missing separate debuginfos, use: yum debuginfo-install postgresql12-server-12.4-1PGDG.rhel8.x86_64
(gdb) bt
#0 0x00007fdbd508b70f in ?? ()
Backtrace stopped: Cannot access memory at address 0x7ffd0e7285c0
(gdb) quit

Any ideas ?

Lars