Re: New Copy Formats - avro/orc/parquet

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Andres Freund <andres(at)anarazel(dot)de>
Cc: Nicolas Paris <niparisco(at)gmail(dot)com>, pgsql-general(at)postgresql(dot)org
Subject: Re: New Copy Formats - avro/orc/parquet
Date: 2018-02-11 23:27:12
Message-ID: 25421.1518391632@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Andres Freund <andres(at)anarazel(dot)de> writes:
> On February 11, 2018 2:48:13 PM PST, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> (Any such patch should manage
>> to turn COPY-CSV into an extension, at least so far as copy.c is
>> concerned, even if we don't package it as one.)

> Yea, I was thinking we should move all three (default, csv, binary)
> supported formats to using such a facility.

Hm, yeah, probably. That opens up a different dimension of variation:
is the raw data text or binary? I'm thinking we shouldn't make the format
handlers reinvent e.g. encoding conversion if they don't have to, so it's
likely that some of the infrastructure for text data would remain in core
copy.c.

regards, tom lane

In response to

Browse pgsql-general by date

  From Date Subject
Next Message Sand Stone 2018-02-12 01:14:38 Fwd: persistent read cache
Previous Message Clive Anglin 2018-02-11 23:10:00 RE: Migrating Postgresql from Linux x86