Re: Segmentation Fault

Turning JIT off seems to have solved the problem so far. But I've gone a
couple of hours w/o it happening and then it starts again. So I'm still
kind of focused on trying to get a core dump.

Here are some of my notes from trying to get one last night...

I am typically using systemctl to launch postgres but have also tested
running manually. I will try adding the ulimit call before running manually
on my next set of tests if I get another failure. If I'm not getting
another failure then I'll re-enable JIT and try to induce it again.

I have 32GB of shared mem configured for postgresql but only 10G of drive
space on my OS drive. I'm wondering if that's causing it to not be able to
write the core dump. I'm going to move my core dump location to my data
drive which has well over 100GB free...

When running systemctl list I notice that I have versions 12 and 14 of
postgresql also running. I would hope that doesn't cause an issue but it
also doesn't seem ideal. How do I remove them from systemd startup?

Running the systemctl show I got the following info...
LimitCPU=infinity
LimitCPUSoft=infinity
LimitFSIZE=infinity
LimitFSIZESoft=infinity
LimitDATA=infinity
LimitDATASoft=infinity
LimitSTACK=infinity
LimitSTACKSoft=8388608
LimitCORE=infinity
LimitCORESoft=0
LimitRSS=infinity
LimitRSSSoft=infinity
LimitNOFILE=524288
LimitNOFILESoft=1024

My pgdata directory is set in my postgresql.conf file as
data_directory = '/var/lib/postgresql/15/main'

PGDATA is not set in the environment.

My core_pattern is set to "/tmp/core-%e-%s-%u-%g-%p-%t"

The contents of that directory are
azureuser(at)kpidb-dev:/etc$ sudo ls -al /var/lib/postgresql/15/main
total 120
drwx------ 20 postgres postgres 4096 Sep 14 19:13 .
drwxr-xr-x 3 postgres postgres 4096 Aug 18 20:44 ..
-rw------- 1 postgres postgres 3 Aug 18 20:44 PG_VERSION
drwx------ 5 postgres postgres 4096 Aug 18 20:44 base
-rw------- 1 postgres postgres 44 Sep 14 19:13 current_logfiles
drwx------ 2 postgres postgres 4096 Sep 14 19:39 global
drwx------ 2 postgres postgres 4096 Sep 14 19:13 log
drwx------ 2 postgres postgres 4096 Aug 18 20:44 pg_commit_ts
drwx------ 2 postgres postgres 4096 Aug 18 20:44 pg_dynshmem
drwx------ 4 postgres postgres 4096 Sep 14 19:39 pg_logical
drwx------ 4 postgres postgres 4096 Aug 18 20:44 pg_multixact
drwx------ 2 postgres postgres 4096 Aug 18 20:44 pg_notify
drwx------ 2 postgres postgres 4096 Aug 18 20:44 pg_replslot
drwx------ 2 postgres postgres 4096 Aug 18 20:44 pg_serial
drwx------ 2 postgres postgres 4096 Aug 18 20:44 pg_snapshots
drwx------ 2 postgres postgres 4096 Sep 14 19:13 pg_stat
drwx------ 2 postgres postgres 4096 Aug 29 01:44 pg_stat_tmp
drwx------ 2 postgres postgres 4096 Sep 5 14:36 pg_subtrans
drwx------ 2 postgres postgres 4096 Aug 30 14:05 pg_tblspc
drwx------ 2 postgres postgres 4096 Aug 18 20:44 pg_twophase
drwx------ 3 postgres postgres 20480 Sep 13 20:16 pg_wal
drwx------ 2 postgres postgres 4096 Aug 18 20:44 pg_xact
-rw------- 1 postgres postgres 88 Aug 18 20:44 postgresql.auto.conf
-rw------- 1 postgres postgres 238 Sep 14 19:13 postmaster.opts
-rw------- 1 postgres postgres 101 Sep 14 19:39 postmaster.pid

On Thu, Sep 14, 2023 at 7:17 PM Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:

> Robert Sanford <wobbet(at)gmail(dot)com> writes:
> > I'm trying to get a core dump but I can't seem to actually generate one.
> > ...
> > I then restarted the postgres server. When I look at the limits in the
> pid
> > folder I see...
> > Limit Soft Limit Hard Limit Units
> > Max cpu time unlimited unlimited
> seconds
> > Max file size unlimited unlimited bytes
> > Max data size unlimited unlimited bytes
> > Max stack size 8388608 unlimited bytes
> > Max core file size 0 unlimited bytes
>
> This shows that you are now allowed to set "ulimit -c unlimited",
> but you have not actually done so, at least not in the session that
> you're using to inspect these settings.
>
> If you are starting the server "by hand" (from a manual shell
> command), it should work to do "ulimit -c unlimited" before issuing
> the command to start the server. If you are using infrastructure such
> as systemd or initscripts, then you'll need to figure out how to get
> it to set the ulimit in the environment of the server process.
>
> BTW, have you tried disabling JIT to see if that stops the crashes?
>
> regards, tom lane
>