Re: Logging parallel worker draught

From: Benoit Lobréau <benoit(dot)lobreau(at)dalibo(dot)com>
To: pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Cc: "Imseih (AWS), Sami" <simseih(at)amazon(dot)com>, Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>, Robert Haas <robertmhaas(at)gmail(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Melanie Plageman <melanieplageman(at)gmail(dot)com>, "Andrey M(dot) Borodin" <x4mmm(at)yandex-team(dot)ru>
Subject: Re: Logging parallel worker draught
Date: 2024-08-28 12:58:51
Message-ID: 8123423a-f041-4f4c-a771-bfd96ab235b0@dalibo.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

Here is a new version of the patch. Sorry for the long delay, I was hit
by a motivation drought and was quite busy otherwise.

The guc is now called `log_parallel_workers` and has three possible values:

* "none": disables logging
* "all": logs parallel worker info for all parallel queries or utilities
* "failure": logs only when the number of parallel workers planned
couldn't be reached.

For this, I added several members to the EState struct.

Each gather node / gather merge node updates the values and the
offending queries are displayed during standard_ExecutorEnd.

For CREATE INDEX / REINDEX on btree and brin, I check the parallel
context struct (pcxt) during _bt_end_parallel() or _brin_end_parallel()
and display a log message when needed.

For vacuum, I do the same in parallel_vacuum_end().

I added some information to the error message for parallel queries as an
experiment. I find it useful, but it can be removed, if you re not
convinced.

2024-08-27 15:59:11.089 CEST [54585] LOG: 1 parallel nodes planned (1
obtained all their workers, 0 obtained none), 2 workers planned (2
workers launched)
2024-08-27 15:59:11.089 CEST [54585] STATEMENT: EXPLAIN (ANALYZE)
SELECT i, avg(j) FROM test_pql GROUP BY i;

2024-08-27 15:59:14.006 CEST [54585] LOG: 2 parallel nodes planned (0
obtained all their workers, 1 obtained none), 4 workers planned (1
workers launched)
2024-08-27 15:59:14.006 CEST [54585] STATEMENT: EXPLAIN (ANALYZE)
SELECT i, avg(j) FROM test_pql GROUP BY i
UNION
SELECT i, avg(j) FROM test_pql GROUP BY i;

For CREATE INDEX / REDINDEX / VACUUMS:

2024-08-27 15:58:59.769 CEST [54521] LOG: 1 workers planned (0 workers
launched)
2024-08-27 15:58:59.769 CEST [54521] STATEMENT: REINDEX TABLE test_pql;

Do you think this is better?

I am not sure if a struct is needed to store the es_nworkers* and if the
modification I did to parallel.h is ok.

Thanks to: Jehan-Guillaume de Rorthais, Guillaume Lelarge and Franck
Boudehen for the help and motivation boost.

(sorry for the spam, I had to resend the mail to the list)

--
Benoit Lobréau
Consultant
http://dalibo.com

Attachment Content-Type Size
0001-Add-logging-for-parallel-worker-usage.patch text/x-patch 12.4 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Robert Haas 2024-08-28 13:00:57 Re: Eager aggregation, take 3
Previous Message Robert Haas 2024-08-28 12:37:16 Re: allowing extensions to control planner behavior