Re: Testing of a fast method to bulk insert a Pandas DataFrame into Postgres

From: Steve Midgley <science(at)misuse(dot)org>
To: Shaozhong SHI <shishaozhong(at)gmail(dot)com>
Cc: pgsql-sql <pgsql-sql(at)lists(dot)postgresql(dot)org>
Subject: Re: Testing of a fast method to bulk insert a Pandas DataFrame into Postgres
Date: 2021-10-04 17:30:18
Message-ID: CAJexoSL6kOb1uhoxoqp6jpMq0+u0DVajuYbYiAZiB6YAMrLuZA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-sql

On Mon, Oct 4, 2021 at 8:55 AM Shaozhong SHI <shishaozhong(at)gmail(dot)com> wrote:

> Has anyone tested this one?
> A Fast Method to Bulk Insert a Pandas DataFrame into Postgres · Ellis
> Valentiner
> <https://ellisvalentiner.com/post/a-fast-method-to-insert-a-pandas-dataframe-into-postgres/>
>
> I tried psql_insert_copy method, but I got the following error message.
>
> to_sql() got an unexpected keyword argument 'method'
>
>
> Can anyone shed light on this?
>
>
Interesting - I wasn't aware there was a faster method than "COPY ... FROM
... CSV"

Maybe my best input is that I've always found COPY/FROM/CSV to be insanely
fast and really easy to write from any standard console or ORM, so maybe
the fractional performance increase from other methods isn't worth it,
given how performant, reliable and easy to use this approach is? Not really
an answer but food for thought..

Steve

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Adrian Klaver 2021-10-04 17:30:22 Re: Testing of a fast method to bulk insert a Pandas DataFrame into Postgres
Previous Message Adrian Klaver 2021-10-04 17:28:34 Re: Testing of a fast method to bulk insert a Pandas DataFrame into Postgres

Browse pgsql-sql by date

  From Date Subject
Next Message Adrian Klaver 2021-10-04 17:30:22 Re: Testing of a fast method to bulk insert a Pandas DataFrame into Postgres
Previous Message Adrian Klaver 2021-10-04 17:28:34 Re: Testing of a fast method to bulk insert a Pandas DataFrame into Postgres