Re: WIP patch: add (PRE|POST)PROCESSOR options to COPY

From: Andrew Dunstan <andrew(at)dunslane(dot)net>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>, Simon Riggs <simon(at)2ndquadrant(dot)com>, Fujii Masao <masao(dot)fujii(at)gmail(dot)com>, Etsuro Fujita <fujita(dot)etsuro(at)lab(dot)ntt(dot)co(dot)jp>, Craig Ringer <ringerc(at)ringerc(dot)id(dot)au>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: WIP patch: add (PRE|POST)PROCESSOR options to COPY
Date: 2012-11-14 17:18:19
Message-ID: 50A3D25B.7070008@dunslane.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers


On 11/14/2012 11:56 AM, Tom Lane wrote:
> Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
>> On 11/14/2012 11:39 AM, Tom Lane wrote:
>>> What happened to the previous proposal of treating the COPY
>>> target as a pipe specification, ie
>> I'd like to be able to filter STDIN if possible - with this syntax how
>> is COPY going to know to hook up STDIN to the program?
> Huh? That's fairly nonsensical for the backend-side case; there's no
> way that stdin (or stdout) of a backend is going to connect anywhere
> useful for this purpose. As for doing it on the psql side (\copy),
> I think it would be more or less automatic. If you do say
>
> foo | psql -c "\copy tab from 'bar |'" dbname
>
> then bar is going to inherit psql's stdin, which is coming from foo.
>
>

Why does it make less sense on the backend than COPY foo FROM STDIN ?
Why shouldn't I want to be able to filter or transform the input? I have
a client with a pretty complex backend-driven ETL tool. One of the
annoying things about it is that we have to transfer the file to the
backend before we can process it. I can imagine this leading to a
similar annoyance.

cheers

andrew

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Bruce Momjian 2012-11-14 17:22:12 Re: Enabling Checksums
Previous Message Tom Lane 2012-11-14 16:56:57 Re: WIP patch: add (PRE|POST)PROCESSOR options to COPY