Segfault in pg_stat_activity

Version info:

Postgres 9.5alpha1 (postgresql-9.5_9.5~alpha1-1.pgdg80+1_amd64.deb)

Distribution: Debian 8.0 (Jessie),
Architecture: AMD64
Libc: 2.19-18

Since I've installed 9.5alpha1 a week ago, I've seen several segfaults
in the log file:

% grep 'server process .* was terminated by signal' *.log
postgresql-2015-07-21_072520.log:2015-07-21 07:26:02 GMT [1600]: [4-1] user=,db=,pid=1600 LOG: server process (PID 25320) was terminated by signal 11: Segmentation fault
postgresql-2015-07-23_082800.log:2015-07-23 11:00:56 GMT [1600]: [9-1] user=,db=,pid=1600 LOG: server process (PID 28864) was terminated by signal 11: Segmentation fault
postgresql-2015-07-23_082800.log:2015-07-23 12:10:58 GMT [1600]: [14-1] user=,db=,pid=1600 LOG: server process (PID 6530) was terminated by signal 11: Segmentation fault
postgresql-2015-07-23_082800.log:2015-07-23 13:05:47 GMT [1600]: [19-1] user=,db=,pid=1600 LOG: server process (PID 14376) was terminated by signal 11: Segmentation fault
postgresql-2015-07-23_134706.log:2015-07-23 13:55:42 GMT [1600]: [24-1] user=,db=,pid=1600 LOG: server process (PID 21482) was terminated by signal 11: Segmentation fault
postgresql-2015-07-24_000000.log:2015-07-24 08:21:10 GMT [1600]: [29-1] user=,db=,pid=1600 LOG: server process (PID 18045) was terminated by signal 11: Segmentation fault
postgresql-2015-07-24_083758.log:2015-07-24 09:05:41 GMT [1600]: [34-1] user=,db=,pid=1600 LOG: server process (PID 24665) was terminated by signal 11: Segmentation fault
postgresql-2015-07-24_093924.log:2015-07-24 09:50:59 GMT [1600]: [39-1] user=,db=,pid=1600 LOG: server process (PID 31328) was terminated by signal 11: Segmentation fault
postgresql-2015-07-24_093924.log:2015-07-24 11:10:55 GMT [1600]: [44-1] user=,db=,pid=1600 LOG: server process (PID 10480) was terminated by signal 11: Segmentation fault
postgresql-2015-07-24_093924.log:2015-07-24 11:50:55 GMT [1600]: [49-1] user=,db=,pid=1600 LOG: server process (PID 16160) was terminated by signal 11: Segmentation fault
postgresql-2015-07-24_093924.log:2015-07-24 11:55:48 GMT [1600]: [54-1] user=,db=,pid=1600 LOG: server process (PID 16947) was terminated by signal 11: Segmentation fault
postgresql-2015-07-24_093924.log:2015-07-24 13:26:02 GMT [1600]: [59-1] user=,db=,pid=1600 LOG: server process (PID 30542) was terminated by signal 11: Segmentation fault
postgresql-2015-07-27_000000.log:2015-07-27 07:55:59 GMT [1600]: [64-1] user=,db=,pid=1600 LOG: server process (PID 22295) was terminated by signal 11: Segmentation fault
postgresql-2015-07-27_115156.log:2015-07-27 15:10:20 GMT [1600]: [69-1] user=,db=,pid=1600 LOG: server process (PID 21239) was terminated by signal 11: Segmentation fault

% grep -A2 'server process .* was terminated by signal' *.log
postgresql-2015-07-21_072520.log:2015-07-21 07:26:02 GMT [1600]: [4-1] user=,db=,pid=1600 LOG: server process (PID 25320) was terminated by signal 11: Segmentation fault
postgresql-2015-07-21_072520.log-2015-07-21 07:26:02 GMT [1600]: [5-1] user=,db=,pid=1600 DETAIL: Failed process was running: SELECT DISTINCT usename,usename FROM pg_stat_activity ORDER BY 1
postgresql-2015-07-21_072520.log-2015-07-21 07:26:02 GMT [1600]: [6-1] user=,db=,pid=1600 LOG: terminating any other active server processes
--
postgresql-2015-07-23_082800.log:2015-07-23 11:00:56 GMT [1600]: [9-1] user=,db=,pid=1600 LOG: server process (PID 28864) was terminated by signal 11: Segmentation fault
postgresql-2015-07-23_082800.log-2015-07-23 11:00:56 GMT [1600]: [10-1] user=,db=,pid=1600 DETAIL: Failed process was running: SELECT DISTINCT usename,usename FROM pg_stat_activity ORDER BY 1
postgresql-2015-07-23_082800.log-2015-07-23 11:00:56 GMT [1600]: [11-1] user=,db=,pid=1600 LOG: terminating any other active server processes
--
postgresql-2015-07-23_082800.log:2015-07-23 12:10:58 GMT [1600]: [14-1] user=,db=,pid=1600 LOG: server process (PID 6530) was terminated by signal 11: Segmentation fault
postgresql-2015-07-23_082800.log-2015-07-23 12:10:58 GMT [1600]: [15-1] user=,db=,pid=1600 DETAIL: Failed process was running: SELECT DISTINCT usename,usename FROM pg_stat_activity ORDER BY 1
postgresql-2015-07-23_082800.log-2015-07-23 12:10:58 GMT [1600]: [16-1] user=,db=,pid=1600 LOG: terminating any other active server processes
[...]
--
postgresql-2015-07-27_000000.log:2015-07-27 07:55:59 GMT [1600]: [64-1] user=,db=,pid=1600 LOG: server process (PID 22295) was terminated by signal 11: Segmentation fault
postgresql-2015-07-27_000000.log-2015-07-27 07:55:59 GMT [1600]: [65-1] user=,db=,pid=1600 DETAIL: Failed process was running: SELECT DISTINCT usename,usename FROM pg_stat_activity ORDER BY 1
postgresql-2015-07-27_000000.log-2015-07-27 07:55:59 GMT [1600]: [66-1] user=,db=,pid=1600 LOG: terminating any other active server processes
--
postgresql-2015-07-27_115156.log:2015-07-27 15:10:20 GMT [1600]: [69-1] user=,db=,pid=1600 LOG: server process (PID 21239) was terminated by signal 11: Segmentation fault
postgresql-2015-07-27_115156.log-2015-07-27 15:10:20 GMT [1600]: [70-1] user=,db=,pid=1600 DETAIL: Failed process was running: SELECT DISTINCT usename,usename FROM pg_stat_activity ORDER BY 1
postgresql-2015-07-27_115156.log-2015-07-27 15:10:20 GMT [1600]: [71-1] user=,db=,pid=1600 LOG: terminating any other active server processes

All the crashed processes were running

"SELECT DISTINCT usename,usename FROM pg_stat_activity ORDER BY 1"

but running that query does not normally produce a crash (as a rough
guess, the chance is about 1%).

Here is the backtrace from the core file produced by the latest crash:

Core was generated by `postgres: postgres template1 [local] SELECT '.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x00007fb94d314289 in memcpy (__len=200, __src=0x7fb948719ad8, __dest=0x7fb94f4c1bb0) at /usr/include/x86_64-linux-gnu/bits/string3.h:51
51 return __builtin___memcpy_chk (__dest, __src, __len, __bos0 (__dest));
(gdb) bt
#0 0x00007fb94d314289 in memcpy (__len=200, __src=0x7fb948719ad8, __dest=0x7fb94f4c1bb0) at /usr/include/x86_64-linux-gnu/bits/string3.h:51
#1 pgstat_read_current_status () at /tmp/buildd/postgresql-9.5-9.5~alpha1/build/../src/backend/postmaster/pgstat.c:3000
#2 0x00007fb94d318db1 in pgstat_read_current_status () at /tmp/buildd/postgresql-9.5-9.5~alpha1/build/../src/backend/postmaster/pgstat.c:2434
#3 pgstat_fetch_stat_numbackends () at /tmp/buildd/postgresql-9.5-9.5~alpha1/build/../src/backend/postmaster/pgstat.c:2435
#4 0x00007fb94d403051 in pg_stat_get_activity (fcinfo=0x7ffe57b2e480) at /tmp/buildd/postgresql-9.5-9.5~alpha1/build/../src/backend/utils/adt/pgstatfuncs.c:534
#5 0x00007fb94d278362 in ExecMakeTableFunctionResult (funcexpr=0x7fb94f37e890, econtext=0x7fb94f37bb10, argContext=<optimized out>, expectedDesc=0x7fb94f380758, randomAccess=0 '\000')
at /tmp/buildd/postgresql-9.5-9.5~alpha1/build/../src/backend/executor/execQual.c:2188
#6 0x00007fb94d28d512 in FunctionNext (node=node(at)entry=0x7fb94f37ba00) at /tmp/buildd/postgresql-9.5-9.5~alpha1/build/../src/backend/executor/nodeFunctionscan.c:94
#7 0x00007fb94d27a539 in ExecScanFetch (recheckMtd=0x7fb94d28d260 <FunctionRecheck>, accessMtd=0x7fb94d28d270 <FunctionNext>, node=0x7fb94f37ba00) at /tmp/buildd/postgresql-9.5-9.5~alpha1/build/../src/backend/executor/execScan.c:82
#8 ExecScan (node=node(at)entry=0x7fb94f37ba00, accessMtd=accessMtd(at)entry=0x7fb94d28d270 <FunctionNext>, recheckMtd=recheckMtd(at)entry=0x7fb94d28d260 <FunctionRecheck>)
at /tmp/buildd/postgresql-9.5-9.5~alpha1/build/../src/backend/executor/execScan.c:132
#9 0x00007fb94d28d568 in ExecFunctionScan (node=node(at)entry=0x7fb94f37ba00) at /tmp/buildd/postgresql-9.5-9.5~alpha1/build/../src/backend/executor/nodeFunctionscan.c:268
#10 0x00007fb94d272d58 in ExecProcNode (node=node(at)entry=0x7fb94f37ba00) at /tmp/buildd/postgresql-9.5-9.5~alpha1/build/../src/backend/executor/execProcnode.c:442
#11 0x00007fb94d285513 in ExecHashJoin (node=node(at)entry=0x7fb94f37ae60) at /tmp/buildd/postgresql-9.5-9.5~alpha1/build/../src/backend/executor/nodeHashjoin.c:154
#12 0x00007fb94d272cd8 in ExecProcNode (node=node(at)entry=0x7fb94f37ae60) at /tmp/buildd/postgresql-9.5-9.5~alpha1/build/../src/backend/executor/execProcnode.c:477
#13 0x00007fb94d28cdc2 in ExecNestLoop (node=node(at)entry=0x7fb94f37ab40) at /tmp/buildd/postgresql-9.5-9.5~alpha1/build/../src/backend/executor/nodeNestloop.c:123
#14 0x00007fb94d272cf8 in ExecProcNode (node=node(at)entry=0x7fb94f37ab40) at /tmp/buildd/postgresql-9.5-9.5~alpha1/build/../src/backend/executor/execProcnode.c:469
#15 0x00007fb94d28f071 in ExecSort (node=node(at)entry=0x7fb94f37a960) at /tmp/buildd/postgresql-9.5-9.5~alpha1/build/../src/backend/executor/nodeSort.c:103
#16 0x00007fb94d272cb8 in ExecProcNode (node=node(at)entry=0x7fb94f37a960) at /tmp/buildd/postgresql-9.5-9.5~alpha1/build/../src/backend/executor/execProcnode.c:488
#17 0x00007fb94d28f325 in ExecUnique (node=node(at)entry=0x7fb94f37a6e0) at /tmp/buildd/postgresql-9.5-9.5~alpha1/build/../src/backend/executor/nodeUnique.c:69
#18 0x00007fb94d272c78 in ExecProcNode (node=node(at)entry=0x7fb94f37a6e0) at /tmp/buildd/postgresql-9.5-9.5~alpha1/build/../src/backend/executor/execProcnode.c:504
#19 0x00007fb94d26fc8e in ExecutePlan (dest=0x7fb94f36ccc8, direction=<optimized out>, numberTuples=0, sendTuples=<optimized out>, operation=CMD_SELECT, planstate=0x7fb94f37a6e0, estate=0x7fb94f37a550)
at /tmp/buildd/postgresql-9.5-9.5~alpha1/build/../src/backend/executor/execMain.c:1549
#20 standard_ExecutorRun (queryDesc=0x7fb94f351a70, direction=<optimized out>, count=0) at /tmp/buildd/postgresql-9.5-9.5~alpha1/build/../src/backend/executor/execMain.c:337
#21 0x00007fb94d37a30f in PortalRunSelect (portal=portal(at)entry=0x7fb94f378540, forward=forward(at)entry=1 '\001', count=0, count(at)entry=9223372036854775807, dest=dest(at)entry=0x7fb94f36ccc8)
at /tmp/buildd/postgresql-9.5-9.5~alpha1/build/../src/backend/tcop/pquery.c:946
#22 0x00007fb94d37b980 in PortalRun (portal=portal(at)entry=0x7fb94f378540, count=count(at)entry=9223372036854775807, isTopLevel=isTopLevel(at)entry=1 '\001', dest=dest(at)entry=0x7fb94f36ccc8, altdest=altdest(at)entry=0x7fb94f36ccc8,
completionTag=completionTag(at)entry=0x7ffe57b2f0c0 "") at /tmp/buildd/postgresql-9.5-9.5~alpha1/build/../src/backend/tcop/pquery.c:790
#23 0x00007fb94d378feb in exec_simple_query (query_string=0x7fb94f3147c0 "SELECT DISTINCT usename,usename FROM pg_stat_activity ORDER BY 1") at /tmp/buildd/postgresql-9.5-9.5~alpha1/build/../src/backend/tcop/postgres.c:1104
#24 PostgresMain (argc=<optimized out>, argv=argv(at)entry=0x7fb94f2bfd70, dbname=0x7fb94f2bfd50 "template1", username=<optimized out>) at /tmp/buildd/postgresql-9.5-9.5~alpha1/build/../src/backend/tcop/postgres.c:4025
#25 0x00007fb94d116ac7 in BackendRun (port=0x7fb94f2c00d0) at /tmp/buildd/postgresql-9.5-9.5~alpha1/build/../src/backend/postmaster/postmaster.c:4159
#26 BackendStartup (port=0x7fb94f2c00d0) at /tmp/buildd/postgresql-9.5-9.5~alpha1/build/../src/backend/postmaster/postmaster.c:3835
#27 ServerLoop () at /tmp/buildd/postgresql-9.5-9.5~alpha1/build/../src/backend/postmaster/postmaster.c:1609
#28 0x00007fb94d31e9b1 in PostmasterMain (argc=5, argv=<optimized out>) at /tmp/buildd/postgresql-9.5-9.5~alpha1/build/../src/backend/postmaster/postmaster.c:1254
#29 0x00007fb94d117c72 in main (argc=5, argv=0x7fb94f27d1e0) at /tmp/buildd/postgresql-9.5-9.5~alpha1/build/../src/backend/main/main.c:221

hp

--
_ | Peter J. Holzer | I want to forget all about both belts and
|_|_) | | suspenders; instead, I want to buy pants
| | | hjp(at)hjp(dot)at | that actually fit.
__/ | http://www.hjp.at/ | -- http://noncombatant.org/