Custom Scans and private data

(sending again, forgot to cc hackers, sorry for the duplicate)

Hi,

I'm trying to use the custom scan API to replace code that currently
does everything via hooks and isn't safe against copyObject() (Yes,
that's not a grand plan).

To me dealing with CustomScan->custom_private seems to be
extraordinarily painful when dealing with nontrivial data
structures. And such aren't all that unlikely when dealing with a custom
scan over something more complex.

Now, custom scans likely modeled private data after FDWs. But it's
already rather painful there (in fact it's the one thing I heard people
complain about repeatedly besides the inability to push down
operations). Just look at the ugly hangups postgres_fdw goes through to
pass data around - storing it in lists with indexes determining the
content and such. That kinda somewhat works if you only integers and
such need to be stored, but if you have more complex data it really is a
PITA. The best alternatives I found are a) to serialize data into a
string, base64 or so. b) use a Const node over a bytea datum. It seems
rather absurd having to go through deserialization at every execution.

Since we already have CustomScan->methods, it seems to be rather
reasonable to have a CopyCustomScan callback and let it do the copying
of the private data if present? Or possibly of the whole node, which'd
allow to embed it into a bigger node?

I looked at pg-strom and it seems to go through rather ugly procedures
to deal with the problem:
https://github.com/pg-strom/devel/blob/master/src/gpujoin.c
form_gpujoin_info/deform_gpujoin_info.

What's the advantage of the current model?

Greetings,

Andres Freund