From: | "Godfrin, Philippe E" <Philippe(dot)Godfrin(at)nov(dot)com> |
---|---|
To: | Gavin Roy <gavinr(at)aweber(dot)com> |
Cc: | "pgsql-general(at)lists(dot)postgresql(dot)org" <pgsql-general(at)lists(dot)postgresql(dot)org> |
Subject: | RE: [EXTERNAL] Re: Inserts and bad performance |
Date: | 2021-11-24 20:27:48 |
Message-ID: | SA0PR15MB3933644C28286B9BEC3B218C82619@SA0PR15MB3933.namprd15.prod.outlook.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-general |
The notion of COPY blocks and asynchronously is very interesting
From: Gavin Roy <gavinr(at)aweber(dot)com>
Sent: Wednesday, November 24, 2021 1:50 PM
To: Godfrin, Philippe E <Philippe(dot)Godfrin(at)nov(dot)com>
Cc: pgsql-general(at)lists(dot)postgresql(dot)org
Subject: [EXTERNAL] Re: Inserts and bad performance
On Wed, Nov 24, 2021 at 2:15 PM Godfrin, Philippe E <Philippe(dot)Godfrin(at)nov(dot)com<mailto:Philippe(dot)Godfrin(at)nov(dot)com>> wrote:GreetingsI am inserting a large number of rows, 5,10, 15 million. The python code commits every 5000 inserts. The table has partitioned children.On the Python client side, if you're usi
On Wed, Nov 24, 2021 at 2:15 PM Godfrin, Philippe E <Philippe(dot)Godfrin(at)nov(dot)com<mailto:Philippe(dot)Godfrin(at)nov(dot)com>> wrote:
Greetings
I am inserting a large number of rows, 5,10, 15 million. The python code commits every 5000 inserts. The table has partitioned children.
On the Python client side, if you're using psycopg, you should consider using using COPY instead of INSERT if you're not:
https://www.psycopg.org/psycopg3/docs/basic/copy.html#copy<https://www.psycopg.org/psycopg3/docs/basic/copy.html#copy>
And if using psycopg2, execute_batch might be of value:
https://www.psycopg.org/docs/extras.html?highlight=insert#psycopg2.extras.execute_batch<https://www.psycopg.org/docs/extras.html?highlight=insert#psycopg2.extras.execute_batch>
Regards,
Gavin
From | Date | Subject | |
---|---|---|---|
Next Message | Michael Lewis | 2021-11-25 00:58:00 | Re: [EXTERNAL] Re: Inserts and bad performance |
Previous Message | Godfrin, Philippe E | 2021-11-24 19:59:38 | RE: [EXTERNAL] Re: Inserts and bad performance |