Re: New Copy Formats - avro/orc/parquet

On 02/11/2018 12:57 PM, Nicolas Paris wrote:
> Le 11 févr. 2018 à 21:53, Andres Freund écrivait :
>> On 2018-02-11 21:41:26 +0100, Nicolas Paris wrote:
>>> I have also the storage and network transfers overhead in mind:
>>> All those new formats are compressed; this is not true for current
>>> postgres BINARY format and obviously text based format. By experience,
>>> the binary format is 10 to 30% larger than the text one. On the
>>> contrary, an ORC file can be up to 10 times smaller than a text base
>>> format.
>>
>> That seems largely irrelevant when arguing about using PROGRAM though,
>> right?
>>
>
> Indeed those storage and network transfers are only considered versus
> CSV/BINARY format. No link with PROGRAM aspect.
>

Just wondering what your time frame is on this? Asking because this
would be considered a new feature and so would need to be added to a
major release of Postgres. Currently work is going on for Postgres
version 11 to be released(just a guess) late Fall 2018/early Winter
2019. The CommitFest(https://commitfest.postgresql.org/) for this
release is currently approximately 3/4 of the way through. Not sure that
new code could make it in at this point. This means it would be bumped
to version 12 for 2019/2020.

--
Adrian Klaver
adrian(dot)klaver(at)aklaver(dot)com