2pc leaks fds

From: Andres Freund <andres(at)anarazel(dot)de>
To: pgsql-hackers(at)postgresql(dot)org, Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>, Antonin Houska <ah(at)cybertec(dot)at>
Subject: 2pc leaks fds
Date: 2020-04-06 02:56:51
Message-ID: 20200406025651.fpzdb5yyb7qyhqko@alap3.anarazel.de
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

Using 2PC with master very quickly leads to:

2020-04-05 19:42:18.368 PDT [2298126][5/2009:0] LOG: out of file descriptors: Too many open files; release and retry
2020-04-05 19:42:18.368 PDT [2298126][5/2009:0] STATEMENT: COMMIT PREPARED 'ptx_2';

This started with:

commit 0dc8ead46363fec6f621a12c7e1f889ba73b55a9 (HEAD -> master)
Author: Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>
Date: 2019-11-25 15:04:54 -0300

Refactor WAL file-reading code into WALRead()

I found this while trying to benchmark the effect of my snapshot changes
on 2pc. I just used the attached pgbench file.

andres(at)awork3:~/build/postgres/dev-assert/vpath$ pgbench -n -s 500 -c 4 -j 4 -T 100000 -P1 -f ~/tmp/pgbench-write-2pc.sql
progress: 1.0 s, 3723.8 tps, lat 1.068 ms stddev 0.305
client 2 script 0 aborted in command 8 query 0: ERROR: could not seek to end of file "base/14036/16396": Too many open files
client 1 script 0 aborted in command 8 query 0: ERROR: could not seek to end of file "base/14036/16396": Too many open files
client 3 script 0 aborted in command 8 query 0: ERROR: could not seek to end of file "base/14036/16396": Too many open files
client 0 script 0 aborted in command 8 query 0: ERROR: could not seek to end of file "base/14036/16396": Too many open files
transaction type: /home/andres/tmp/pgbench-write-2pc.sql

I've not yet reviewed the change sufficiently to pinpoint the issue.

It's a bit sad that nobody has hit this in the last few months :(.

Greetings,

Andres Freund

Attachment Content-Type Size
pgbench-write-2pc.sql application/x-sql 715 bytes

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Amit Kapila 2020-04-06 03:25:01 Re: WAL usage calculation patch
Previous Message Michael Paquier 2020-04-06 02:47:57 Re: ALTER tbl rewrite loses CLUSTER ON index