Converting SetOp to read its two inputs separately

Here are the beginnings of a patchset to do what was discussed at [1],
namely change the SetOp node type to read its inputs as outerPlan and
innerPlan, rather than appending them together with a flag column to
show which rows came from where.

The previous thread wondered why manually DISTINCT'ing inputs with a
lot of duplicates would make an INTERSECT faster than not doing so.
I speculated that the overhead of attaching the flag column (which
typically requires an additional projection node) plus the overhead of
an Append node might have a lot to do with that, and it seems I was
right. With this patchset on yesterday's HEAD, I get results like
these for a simple test case:

regression=# create table t1 as
regression-# select (random()*100)::int as a from generate_series(1,1000000);
SELECT 1000000
regression=# vacuum analyze t1;
VACUUM
regression=# set max_parallel_workers_per_gather TO 0;
SET
regression=# explain analyze select a from t1 intersect select a from t1;
QUERY PLAN
------------------------------------------------------------------------------------------------------------------------
HashSetOp Intersect (cost=0.00..33960.00 rows=101 width=4) (actual time=246.352..246.355 rows=101 loops=1)
-> Seq Scan on t1 (cost=0.00..14480.00 rows=1000000 width=4) (actual time=0.025..35.646 rows=1000000 loops=1)
-> Seq Scan on t1 t1_1 (cost=0.00..14480.00 rows=1000000 width=4) (actual time=0.011..36.163 rows=1000000 loops=1)
Planning Time: 0.045 ms
Execution Time: 246.372 ms
(5 rows)

regression=# explain analyze select distinct a from t1 intersect select distinct a from t1;
QUERY PLAN
------------------------------------------------------------------------------------------------------------------------------
HashSetOp Intersect (cost=33960.00..33962.52 rows=101 width=4) (actual time=238.888..238.891 rows=101 loops=1)
-> HashAggregate (cost=16980.00..16981.01 rows=101 width=4) (actual time=120.749..120.756 rows=101 loops=1)
Group Key: t1.a
Batches: 1 Memory Usage: 24kB
-> Seq Scan on t1 (cost=0.00..14480.00 rows=1000000 width=4) (actual time=0.027..35.391 rows=1000000 loops=1)
-> HashAggregate (cost=16980.00..16981.01 rows=101 width=4) (actual time=118.101..118.107 rows=101 loops=1)
Group Key: t1_1.a
Batches: 1 Memory Usage: 24kB
-> Seq Scan on t1 t1_1 (cost=0.00..14480.00 rows=1000000 width=4) (actual time=0.014..35.468 rows=1000000 loops=1)
Planning Time: 0.043 ms
Execution Time: 238.916 ms
(11 rows)

It's still a little slower without DISTINCT, but not 50% slower like
before. I have hopes that it'll be nearly on par once I figure out
how to avoid the extra per-row slot type conversion that's being done
in the 0001 patch.

Aside from that minor TODO, the main thing that's left undone in this
patch series is to persuade the thing to exploit presorted input
paths. Right now it fails to do so, as can be seen in some of the
regression test cases, eg

regression=# set enable_hashagg = 0;
SET
regression=# explain (costs off) select unique1 from tenk1 except select unique2 from tenk1 where unique2 != 10;
QUERY PLAN
------------------------------------------------------------------
SetOp Except
-> Sort
Sort Key: tenk1.unique1
-> Index Only Scan using tenk1_unique1 on tenk1
-> Sort
Sort Key: tenk1_1.unique2
-> Index Only Scan using tenk1_unique2 on tenk1 tenk1_1
Filter: (unique2 <> 10)
(8 rows)

Obviously the sorts are unnecessary, but it's not seeing that.
I suppose this requires integrating generate_nonunion_paths with
the logic from commit 66c0185a3. I tried to make sense of that,
but failed --- either I don't understand it, or there are a
number of things wrong with it. I'd welcome some help with
getting that done.

regards, tom lane

[1] https://www.postgresql.org/message-id/2631313.1730733484%40sss.pgh.pa.us