From: | Sutou Kouhei <kou(at)clear-code(dot)com> |
---|---|
To: | sushrut(dot)shivaswamy(at)gmail(dot)com |
Cc: | pgsql-hackers(at)lists(dot)postgresql(dot)org |
Subject: | Re: Columnar format export in Postgres |
Date: | 2024-06-13 00:21:34 |
Message-ID: | 20240613.092134.184277386367461532.kou@clear-code.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
Hi,
In <CAH5mb9_YW76_0xBU2T4C7HF33b+b2w3QBtV50_ZZMy8SA8ChjA(at)mail(dot)gmail(dot)com>
"Columnar format export in Postgres" on Wed, 12 Jun 2024 22:26:30 +0530,
Sushrut Shivaswamy <sushrut(dot)shivaswamy(at)gmail(dot)com> wrote:
> I have been working on adding support for columnar format export to
> Postgres to speed up analytics queries.
FYI: I'm proposing making COPY format extendable:
* https://www.postgresql.org/message-id/flat/20231204(dot)153548(dot)2126325458835528809(dot)kou(at)clear-code(dot)com
* https://commitfest.postgresql.org/48/4681/
If it's accepted, we can implement extensions for COPY
FORMAT arrow and COPY FORMAT parquet. With these extensions,
we can use file_fdw to read Apache Arrow and Apache Parquet
file because file_fdw is based on COPY FROM:
https://www.postgresql.org/docs/current/file-fdw.html
If you're interested in this proposal, you can review the
latest proposed patch set to proceed this proposal.
> - Reduce memory consumption when exporting table data to columnar format
The above COPY support will help this.
Thanks,
--
kou
From | Date | Subject | |
---|---|---|---|
Next Message | Tatsuo Ishii | 2024-06-13 00:25:01 | Re: Row pattern recognition |
Previous Message | Thomas Munro | 2024-06-12 23:11:56 | Changing default -march landscape |