Re: WIP patch: add (PRE|POST)PROCESSOR options to COPY

> From: Tom Lane [mailto:tgl(at)sss(dot)pgh(dot)pa(dot)us]

> "Etsuro Fujita" <fujita(dot)etsuro(at)lab(dot)ntt(dot)co(dot)jp> writes:
> > Maybe my explanation was insufficient. Let me add one thing to my earlier
> > explanation. The submitted patch allows the psql \copy instruction to be
> > executed like:
>
> > $ echo '/bin/gunzip -c $1' > decompress.sh
> > $ chmod +x decompress.sh
>
> > postgres=# \copy foo FROM '/home/pgsql/foo.csv.gz' WITH (format 'csv',
> > preprocessor '/home/pgsql/decompress.sh')

> Well, in that case, you've got not only an explanation problem but a
> syntax problem, because that syntax is utterly misleading. Anybody
> looking at it would think that the "format" option is one of the options
> being sent to the backend. The code required to pull it out of there
> has got to be grossly overcomplicated (and likely bugprone), too.
>
> I think it would be better to present this as something like
>
> \copy foo from '/home/pgsql/decompress.sh /home/pgsql/foo.csv.gz |' with
> format 'csv'
>
> which would cue any reasonably Unix-savvy person that what's happening
> is a popen on the client side. It'd probably be a whole lot less
> complicated to implement, too.

Great!

I have a question. I think it would be also better to extend the syntax for the
SQL COPY command in the same way, ie,

COPY foo from '/home/pgsql/decompress.sh /home/pgsql/foo.csv.gz |' with format
'csv'

Is this okay?

Thanks,

Best regards,
Etsuro Fujita