pgbench - rework variable management

From: Fabien COELHO <coelho(at)cri(dot)ensmp(dot)fr>
To: PostgreSQL Developers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Cc: PostgreSQL Developers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: pgbench - rework variable management
Date: 2019-08-13 15:54:31
Message-ID: alpine.DEB.2.21.1908131745380.6526@lancre
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers


Hello pgdevs,

The attached patch improves pgbench variable management as discussed in:

https://www.postgresql.org/message-id/flat/alpine(dot)DEB(dot)2(dot)21(dot)1904081752210(dot)5867(at)lancre

As discussed there as well, the overall effect is small compared to libpq
& system costs when pgbench is talking to a postgres server. When someone
says "pgbench is slow", they really mean "libpq & <my-system> are slow",
because pgbench does not do much beyond jumping from one libpq call to the
next. Anyway, the patch has a measurable positive effect.

###

Rework pgbench variables and associated values for better performance

- a (hopefully) thread-safe symbol table which maps variable names to integers
note that all variables are statically known, but \gset stuff.
- numbers are then used to access per-client arrays

The symbol table stores names as distinct leaves in a tree on bytes.
Each symbol name is the shortest-prefix leaf, possibly including the final
'\0'. Some windows-specific hacks are note tested. File "symbol_table_test.c"
does what it says and can be compiled standalone.

Most malloc/free cycles are taken out of running a benchmark:
- there is a (large?) maximum number of variables of 32*MAX_SCRIPTS
- variable names and string values are statically allocated,
and limited to, 64 bytes
- a per-client persistent buffer is used for various purpose,
to avoid mallocs/frees.

Functions assignVariables & parseQuery basically shared the same variable
substitution logic, but differed in what was substituted. The logic has been
abstracted into a common function.

This patch brings pgbench-specific overheads down on some tests, one
thread one client, on my laptop, with the attached scripts, in tps:
- set_x_1.sql: 11.1M -> 14.2M
- sets.sql: 0.8M -> 2.7M # 20 \set
- set.sql: 1.5M -> 2.0M # 3 \set & "complex" expressions
- empty.sql: 63.9K -> 64.1K (…)
- select_aid.sql: 29.3K -> 29.3K
- select_aids.sql: 23.4K -> 24.2K
- gset_aid.sql: 28.3K -> 29.2K

So we are talking significant improvements on pgbench-only scripts, only
a few percents once pgbench must interact with a CPU-bound server, because
time is spent elsewhere.

--
Fabien.

Attachment Content-Type Size
empty.sql application/x-sql 2 bytes
select_aids.sql application/x-sql 77 bytes
select_aid.sql application/x-sql 70 bytes
pgbench-symbols-2.patch text/x-diff 49.1 KB
set_x_1.sql application/x-sql 9 bytes
sets.sql application/x-sql 212 bytes
gset_aid.sql application/x-sql 64 bytes

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2019-08-13 15:58:52 Re: SegFault on 9.6.14
Previous Message Anastasia Lubennikova 2019-08-13 15:45:15 Re: [HACKERS] [WIP] Effective storage of duplicates in B-tree index.