Re: Make COPY extendable in order to support Parquet and other formats

From: Ashutosh Bapat <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com>
To: Aleksander Alekseev <aleksander(at)timescale(dot)com>
Cc: PostgreSQL Development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Make COPY extendable in order to support Parquet and other formats
Date: 2022-06-21 09:38:27
Message-ID: CAExHW5u5ntTeOen8RDoX6G9X9McZeDyi-jzJTfZSSiCjhJcn4Q@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Mon, Jun 20, 2022 at 8:35 PM Aleksander Alekseev
<aleksander(at)timescale(dot)com> wrote:
>
> I would like to invest some time into providing a corresponding patch
> for the core and implementing "pg_copy_parquet" extension as a
> practical example, and yet another, a bit simpler, extension as an API
> usage example for the core codebase. I just wanted to double-check
> that this is still a wanted feature and no one on pgsql-hackers@
> objects the idea.

An extension just for COPY to/from parquet looks limited in
functionality. Shouldn't this be viewed as an FDW or Table AM support
for parquet or other formats? Of course the later is much larger in
scope compared to the first one. But there may already be efforts
underway [1]
https://www.postgresql.org/about/news/parquet-s3-fdw-01-was-newly-released-2179/

I have not used it myself or worked with it.

--
Best Wishes,
Ashutosh Bapat

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Magnus Hagander 2022-06-21 09:43:40 Re: Devel docs on website reloading
Previous Message Amit Kapila 2022-06-21 09:35:13 Re: Use fadvise in wal replay