From: | Imre Samu <pella(dot)samu(at)gmail(dot)com> |
---|---|
To: | Softwarelimits <softwarelimits(at)gmail(dot)com> |
Cc: | "pgsql-generallists(dot)postgresql(dot)org" <pgsql-general(at)lists(dot)postgresql(dot)org> |
Subject: | Re: How to import Apache parquet files? |
Date: | 2019-11-05 15:09:42 |
Message-ID: | CAJnEWwnBk1qACWNBN2NkvZ7uewFPCQ2fkLEifT-qQEHLkb8vmg@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-general |
>I would like to import (lots of) Apache parquet files to a PostgreSQL 11
cluster
imho: You have to check and test the Parquet FDW ( Parquet File Wrapper )
- https://github.com/adjust/parquet_fdw
Imre
Softwarelimits <softwarelimits(at)gmail(dot)com> ezt írta (időpont: 2019. nov. 5.,
K, 15:57):
> Hi, I need to come and ask here, I did not find enough information so I
> hope I am just having a bad day or somebody is censoring my search results
> for fun... :)
>
> I would like to import (lots of) Apache parquet files to a PostgreSQL 11
> cluster - yes, I believe it should be done with the Python pyarrow module,
> but before digging into the possible traps I would like to ask here if
> there is some common, well understood and documented tool that may be
> helpful with that process?
>
> It seems that the COPY command can import binary data, but I am not able
> to allocate enough resources to understand how to implement a parquet file
> import with that.
>
> I really would like follow a person with much more knowledge than me about
> either PostgreSQL or Apache parquet format instead of inventing a bad
> wheel.
>
> Any hints very welcome,
> thank you very much for your attention!
> John
>
From | Date | Subject | |
---|---|---|---|
Next Message | Softwarelimits | 2019-11-05 15:21:45 | Re: How to import Apache parquet files? |
Previous Message | Alvaro Herrera | 2019-11-05 15:07:39 | Re: v12 and pg_restore -f- |