Re: Showing applied extended statistics in explain Part 2

Hi All,

I've organized the discussion so far and improved the patch.
The issues and their status are below.

* Issues and status (or comment):

I've numbered them for ease of management.
====
T6. Changed option to show extended statistics (from VERBOSE to STATS)
-> Done already on the previous patch

T4. Add new node (to resolve errors in cfbot and prepared statement)
-> Done on the attached patch maybe

M2. Handle VERBOSE option more correctly
-> Done on the attache patch

M4. Organize the specifications for the types of ExtStats to display
-> Done. I understood the specifications (restrictions) below and wrote
it
on explain.sgml.
- depndencied or mcv of extended stats are displayed
- ndistinct is excluded

T9. Add log_stats option to auto_explain
-> Done on the attached patches

I2. Add tab completion to psql
-> Done on the attached patches

T7. Add documents
-> Partially completed.
- Done:
- doc/src/sgml/ref/explain.sgml
- doc/src/sgml/auto-explain.sgml
- Not started yet:
- doc/src/sgml/ref/create_statistics.sgml
- doc/src/sgml/perform.sgml
- doc/src/sgml/planstats.sgml

T8. Add regression test (stats_ext.sql)
-> Work in progress.

M5. Extstats showing twice issue
-> Work in progress.
The problem is related to Partial/Final groupby or Parallel query.
I found that HashParallel is one of the causes to show it twice.
(See result.txt)

I1. Fix error when query has nested OR condition
-> Not started yet.
As a starting point, it's good to understand the difference
between maybe_extract_actual_clauses() and extract_actual_clauses(),
right?

T5. Measure overhead of the feature
-> Not started yet.
But I understood a conditions to measure overhead:
- simple query without no extended stats
- simple query with 100 extended stats
====

* Regarding to the attached patches:

This patch is WIP patch including the following fixes:
====
- 0001-Add-a-new-option-STATS-to-EXPLAIN-command-r3.patch
- Rebased on 9926f854
- Added a new struct Applied_ExtStats in plannode.h (T4)
- Hopefully this will solve the issue related to
"-DCOPY_PARSE_PLAN_TREES"
with cfbot.
- To pass the extended statistics information from path to plan,
it might be more appropriate to define a new structure in
primnode.h
rather than plannode.h. Any advice would be appreciated.
- Handled EXPLAIN(STATS, VERBOSE) option (M2)
- Before the fix, schema name was always added to extended
statistics name,
but with this patch, schema name is added to the following only
when
VERBOSE option is selected:
- Extended statistics name, table name, and column name
- Added Supported extended statistics types in document (M4)

- 0002-Add-a-new-option-auto_explain.log_stats.patch
- It allows to use auto_explain.log_stats option on auto_explain

- 0003-Add-a-new-tab-completion-for-EXPLAIN-STATS-on-psql.patch
- When you run "EXPLAIN (<tab>" or "EXPLAIN (S<tab>" on psql,
"STATS" string is displayed.
=====

Finally, the attached test.sql and result.txt are a test case (incomplete)
and
its results. When the test case is completed, it will be merged into
ext_stats.sql.

Any advice is welcome!

Regards,
Tatsuro Yamada