| From: | Chris Travers <chris(dot)travers(at)gmail(dot)com> |
|---|---|
| To: | Scott Ribe <scott_ribe(at)elevated-dev(dot)com> |
| Cc: | PostgreSQL General <pgsql-general(at)lists(dot)postgresql(dot)org> |
| Subject: | Re: export to parquet |
| Date: | 2020-08-26 19:11:13 |
| Message-ID: | CAKt_ZfvxgX4ZO5r0hWrRzP9b+X9cnVVshk_CZ=4kX-y5AG5Wew@mail.gmail.com |
| Views: | Raw Message | Whole Thread | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-general |
On Wed, Aug 26, 2020 at 9:00 PM Scott Ribe <scott_ribe(at)elevated-dev(dot)com>
wrote:
> I have no Hadoop, no HDFS. Just looking for the easiest way to export some
> PG tables into Parquet format for testing--need to determine what kind of
> space reduction we can get before deciding whether to look into it more.
>
> Any suggestions on particular tools? (PG 12, Linux)
>
> For simple exporting, the simplest thing is a single-node instance of
Spark.
You can read parquet files in Postgres using
https://github.com/adjust/parquet_fdw if you so desire but it does not
support writing as parquet files are basically immutable.
>
> --
> Scott Ribe
> scott_ribe(at)elevated-dev(dot)com
> https://www.linkedin.com/in/scottribe/
>
>
>
>
>
>
--
Best Wishes,
Chris Travers
Efficito: Hosted Accounting and ERP. Robust and Flexible. No vendor
lock-in.
http://www.efficito.com/learn_more
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Scott Ribe | 2020-08-26 19:29:48 | Re: export to parquet |
| Previous Message | Scott Ribe | 2020-08-26 19:00:21 | export to parquet |