Re: New Copy Formats - avro/orc/parquet

On 02/10/2018 04:38 PM, David G. Johnston wrote:
> On Saturday, February 10, 2018, Nicolas Paris <niparisco(at)gmail(dot)com
> <mailto:niparisco(at)gmail(dot)com>> wrote:
>
> Hello
>
> I d'found useful to be able to import/export from postgres to those
> modern data
> formats:
> - avro (c writer=https://avro.apache.org/docs/1.8.2/api/c/index.html
> <https://avro.apache.org/docs/1.8.2/api/c/index.html>)
> - parquet (c++ writer=https://github.com/apache/parquet-cpp
> <https://github.com/apache/parquet-cpp>)
> - orc (all writers=https://github.com/apache/orc
> <https://github.com/apache/orc>)
>
> Something like :
> COPY table TO STDOUT ORC;
>
> Would be lovely.
>
> This would greatly enhance how postgres integrates in big-data
> ecosystem.
>
>  
> It would be written "... with (format 'orc')" and your best bet would be
> to create an extension.  I don't think that having such code in core (or
> contrib) is desirable.
>

I don't think make this extensible by an extension (i.e. the formats
supported by COPY are hard-coded in core). But I agree that if we are to
add multiple new formats, it'd be nice to allow doing that in extension.

regards

--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services