Re: WIP patch: add (PRE|POST)PROCESSOR options to COPY

"Etsuro Fujita" <fujita(dot)etsuro(at)lab(dot)ntt(dot)co(dot)jp> writes:
> Maybe my explanation was insufficient. Let me add one thing to my earlier
> explanation. The submitted patch allows the psql \copy instruction to be
> executed like:

> $ echo '/bin/gunzip -c $1' > decompress.sh
> $ chmod +x decompress.sh

> postgres=# \copy foo FROM '/home/pgsql/foo.csv.gz' WITH (format 'csv',
> preprocessor '/home/pgsql/decompress.sh')

> In this example, command "/home/pgsql/decompress.sh /home/pgsql/foo.csv.gz" is
> executed on client side, by using popen(), and command "COPY foo FROM STDIN WITH
> (format 'csv')" is sent to backend. I apologize for not providing you with
> enough explanation.

Well, in that case, you've got not only an explanation problem but a
syntax problem, because that syntax is utterly misleading. Anybody
looking at it would think that the "format" option is one of the options
being sent to the backend. The code required to pull it out of there
has got to be grossly overcomplicated (and likely bugprone), too.

I think it would be better to present this as something like

\copy foo from '/home/pgsql/decompress.sh /home/pgsql/foo.csv.gz |' with format 'csv'

which would cue any reasonably Unix-savvy person that what's happening
is a popen on the client side. It'd probably be a whole lot less
complicated to implement, too.

regards, tom lane