From: | Fabien COELHO <coelho(at)cri(dot)ensmp(dot)fr> |
---|---|
To: | PostgreSQL Developers <pgsql-hackers(at)lists(dot)postgresql(dot)org> |
Cc: | PostgreSQL Developers <pgsql-hackers(at)lists(dot)postgresql(dot)org> |
Subject: | pgbench - rework variable management |
Date: | 2019-08-13 15:54:31 |
Message-ID: | alpine.DEB.2.21.1908131745380.6526@lancre |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
Hello pgdevs,
The attached patch improves pgbench variable management as discussed in:
As discussed there as well, the overall effect is small compared to libpq
& system costs when pgbench is talking to a postgres server. When someone
says "pgbench is slow", they really mean "libpq & <my-system> are slow",
because pgbench does not do much beyond jumping from one libpq call to the
next. Anyway, the patch has a measurable positive effect.
###
Rework pgbench variables and associated values for better performance
- a (hopefully) thread-safe symbol table which maps variable names to integers
note that all variables are statically known, but \gset stuff.
- numbers are then used to access per-client arrays
The symbol table stores names as distinct leaves in a tree on bytes.
Each symbol name is the shortest-prefix leaf, possibly including the final
'\0'. Some windows-specific hacks are note tested. File "symbol_table_test.c"
does what it says and can be compiled standalone.
Most malloc/free cycles are taken out of running a benchmark:
- there is a (large?) maximum number of variables of 32*MAX_SCRIPTS
- variable names and string values are statically allocated,
and limited to, 64 bytes
- a per-client persistent buffer is used for various purpose,
to avoid mallocs/frees.
Functions assignVariables & parseQuery basically shared the same variable
substitution logic, but differed in what was substituted. The logic has been
abstracted into a common function.
This patch brings pgbench-specific overheads down on some tests, one
thread one client, on my laptop, with the attached scripts, in tps:
- set_x_1.sql: 11.1M -> 14.2M
- sets.sql: 0.8M -> 2.7M # 20 \set
- set.sql: 1.5M -> 2.0M # 3 \set & "complex" expressions
- empty.sql: 63.9K -> 64.1K (…)
- select_aid.sql: 29.3K -> 29.3K
- select_aids.sql: 23.4K -> 24.2K
- gset_aid.sql: 28.3K -> 29.2K
So we are talking significant improvements on pgbench-only scripts, only
a few percents once pgbench must interact with a CPU-bound server, because
time is spent elsewhere.
--
Fabien.
Attachment | Content-Type | Size |
---|---|---|
empty.sql | application/x-sql | 2 bytes |
select_aids.sql | application/x-sql | 77 bytes |
select_aid.sql | application/x-sql | 70 bytes |
pgbench-symbols-2.patch | text/x-diff | 49.1 KB |
set_x_1.sql | application/x-sql | 9 bytes |
sets.sql | application/x-sql | 212 bytes |
gset_aid.sql | application/x-sql | 64 bytes |
From | Date | Subject | |
---|---|---|---|
Next Message | Tom Lane | 2019-08-13 15:58:52 | Re: SegFault on 9.6.14 |
Previous Message | Anastasia Lubennikova | 2019-08-13 15:45:15 | Re: [HACKERS] [WIP] Effective storage of duplicates in B-tree index. |