Re: Testing of a fast method to bulk insert a Pandas DataFrame into Postgres

Hello, Steve,

That is interesting.

Send me the link for Python doing "Copy---From....CSV".

I would like to review and test.

Regards,

David

On Mon, 4 Oct 2021 at 18:30, Steve Midgley <science(at)misuse(dot)org> wrote:

>
>
> On Mon, Oct 4, 2021 at 8:55 AM Shaozhong SHI <shishaozhong(at)gmail(dot)com>
> wrote:
>
>> Has anyone tested this one?
>> A Fast Method to Bulk Insert a Pandas DataFrame into Postgres · Ellis
>> Valentiner
>> <https://ellisvalentiner.com/post/a-fast-method-to-insert-a-pandas-dataframe-into-postgres/>
>>
>> I tried psql_insert_copy method, but I got the following error message.
>>
>> to_sql() got an unexpected keyword argument 'method'
>>
>>
>> Can anyone shed light on this?
>>
>>
> Interesting - I wasn't aware there was a faster method than "COPY ... FROM
> ... CSV"
>
> Maybe my best input is that I've always found COPY/FROM/CSV to be insanely
> fast and really easy to write from any standard console or ORM, so maybe
> the fractional performance increase from other methods isn't worth it,
> given how performant, reliable and easy to use this approach is? Not really
> an answer but food for thought..
>
> Steve
>
>