pgsql: Get rid of O(N^2) script-parsing overhead in pgbench.

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: pgsql-committers(at)lists(dot)postgresql(dot)org
Subject: pgsql: Get rid of O(N^2) script-parsing overhead in pgbench.
Date: 2025-02-27 15:58:27
Message-ID: E1tngHL-0000O9-1I@gemulon.postgresql.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-committers

Get rid of O(N^2) script-parsing overhead in pgbench.

pgbench wants to record the starting line number of each command
in its scripts. It was computing that by scanning from the script
start and counting newlines, so that O(N^2) work had to be done
for an N-command script. In a script with 50K lines, this adds
up to about 10 seconds on my machine.

To add insult to injury, the results were subtly wrong, because
expr_scanner_offset() scanned to find the NUL that flex inserts
at the end of the current token --- and before the first yylex
call, no such NUL has been inserted. So we ended by computing the
script's last line number not its first one. This was visible only
in case of \gset at the start of a script, which perhaps accounts
for the lack of complaints.

To fix, steal an idea from plpgsql and track the current lexer
ending position and line count as we advance through the script.
(It's a bit simpler than plpgsql since we can't need to back up.)
Also adjust a couple of other places that were invoking scans
from script start when they didn't really need to. I made a new
psqlscan function psql_scan_get_location() that replaces both
expr_scanner_offset() and expr_scanner_get_lineno(), since in
practice expr_scanner_get_lineno() was only being invoked to find
the line number of the current lexer end position.

Reported-by: Daniel Vérité <daniel(at)manitou-mail(dot)org>
Author: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Discussion: https://postgr.es/m/84a8a89e-adb8-47a9-9d34-c13f7150ee45@manitou-mail.org

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/c8c74ad7e1cbc71b616f8ae786776c521729976b

Modified Files
--------------
src/bin/pgbench/exprscan.l | 65 ++++++++++++-------------------------
src/bin/pgbench/pgbench.c | 14 ++++----
src/bin/pgbench/pgbench.h | 4 +--
src/fe_utils/psqlscan.l | 54 ++++++++++++++++++++++++++++++
src/include/fe_utils/psqlscan.h | 3 ++
src/include/fe_utils/psqlscan_int.h | 4 +++
6 files changed, 89 insertions(+), 55 deletions(-)

Browse pgsql-committers by date

  From Date Subject
Next Message Peter Eisentraut 2025-02-27 16:15:46 pgsql: Generalize hash and ordering support in amapi
Previous Message Alexander Korotkov 2025-02-27 09:28:01 pgsql: Get rid of ojrelid local variable in remove_rel_from_query()