From: | Softwarelimits <softwarelimits(at)gmail(dot)com> |
---|---|
To: | "pgsql-generallists(dot)postgresql(dot)org" <pgsql-general(at)lists(dot)postgresql(dot)org> |
Subject: | Re: How to import Apache parquet files? |
Date: | 2019-11-05 15:21:45 |
Message-ID: | CALnJc4XX1cc=oDX8SdKwDTAb5grZPx9Zu02f7LwM5OhmrBPFxw@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-general |
Hi Imre, thanks for the quick response - yes, I found that, but I was not
sure if it is already production ready - also I would like to use the data
with the timescale extension, that is why I need a full import.
Have nice day!
On Tue, Nov 5, 2019 at 4:09 PM Imre Samu <pella(dot)samu(at)gmail(dot)com> wrote:
> >I would like to import (lots of) Apache parquet files to a PostgreSQL 11
> cluster
>
> imho: You have to check and test the Parquet FDW ( Parquet File Wrapper )
> - https://github.com/adjust/parquet_fdw
>
> Imre
>
>
>
>
> Softwarelimits <softwarelimits(at)gmail(dot)com> ezt írta (időpont: 2019. nov.
> 5., K, 15:57):
>
>> Hi, I need to come and ask here, I did not find enough information so I
>> hope I am just having a bad day or somebody is censoring my search results
>> for fun... :)
>>
>> I would like to import (lots of) Apache parquet files to a PostgreSQL 11
>> cluster - yes, I believe it should be done with the Python pyarrow module,
>> but before digging into the possible traps I would like to ask here if
>> there is some common, well understood and documented tool that may be
>> helpful with that process?
>>
>> It seems that the COPY command can import binary data, but I am not able
>> to allocate enough resources to understand how to implement a parquet file
>> import with that.
>>
>> I really would like follow a person with much more knowledge than me
>> about either PostgreSQL or Apache parquet format instead of inventing a bad
>> wheel.
>>
>> Any hints very welcome,
>> thank you very much for your attention!
>> John
>>
>
From | Date | Subject | |
---|---|---|---|
Next Message | Tom Lane | 2019-11-05 15:44:51 | Re: select view definition from pg_views feature request |
Previous Message | Imre Samu | 2019-11-05 15:09:42 | Re: How to import Apache parquet files? |