Re: BUG #18261: Inconsistent results of SELECT affected by joined subqueries

On 2/1/2024 08:43, Richard Guo wrote:
>
> On Fri, Dec 29, 2023 at 12:32 AM Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us
> <mailto:tgl(at)sss(dot)pgh(dot)pa(dot)us>> wrote:
>
> Richard Guo <guofenglinux(at)gmail(dot)com <mailto:guofenglinux(at)gmail(dot)com>>
> writes:
> > I've looked into it a bit.  The problem lies in how the SJE code
> handles
> > the transfer of qual clauses from the removed relation to the
> remaining
> > one.
>
> I am definitely starting to think that the SJE patch was not ready
> for prime time.  We keep finding not only minor but major problems
> in it --- I'd call this one a "major" one.  Is it time to revert
> and rethink it from scratch?
>
>
> I feel the same way.  It seems to me that the SJE code still needs some
> refactoring to get to a committable state.
Being an author of the feature, I couldn't be all objective, of course.
However, let me write some words on this code.

About profitability. According to current database development trends,
the feature looks helpful, where we see many auto-generated and
syntactically redundant queries. So, IMO, some tools for simplifying the
parse tree should be in the pocket of the DBMS. We do have numerous
planner hooks that give way to well-known extensions like Citus or
TimescaleDB, but tweaking the parse tree is gradually cheaper in many cases.
By reducing the number of RangeTblEntries, clauses, and equivalence
classes, we can profit significantly from this feature, especially with
partitioned tables.
Hence, this code is not only about self-join elimination. It also
introduces machinery for redundant parse tree reduction not only for the
case of dead tails, like reduce_outer_joins. It may be utilized by
extensions in other ways, too.

About weak points. We tried different ways to implement that feature
during the development, even in the optimization stage. The current
place and logic are the most effective. The self-join detection stage
looks stable and polished enough. Only one part of the code is debatable
- replacing one baserel (removing), involved in many relations with
other DB objects, with another one (keeping).

About development. Having it working in a PG fork and as a patch at the
commitfest for a long time, we hadn't had so much food for thought as
last three months when it was committed to the master. It raised new
questions on weaknesses of setrefs, links from outer query blocks,
integrity control of PlannerInfo, etc.
Bugs we have been finding look at most not about architectural issues
but about leaks in planner understanding, which is primarily correct
about me personally.
I think, having close scrutiny of the PlannerInfo fields and clause
replacement strategy, we can achieve stable behaviour of this feature
before the release and I already see progress in stability of the
feature. As I see dynamics of discussions in the pgsql-bugs list, we
already have a community team of people who detect SJE problems, develop
the code and provide a review. Maybe it is also a positive outcome?

Having written all the words above, I think we must formulate objective
reasons if we want to revert this feature. Another way it can live on
the mailing list for another five years without any progress. I,
personally, vote for leaving this feature in the master.

--
regards,
Andrei Lepikhov
Postgres Professional