From: | Andres Freund <andres(at)anarazel(dot)de> |
---|---|
To: | pgsql-performance(at)lists(dot)postgresql(dot)org, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, aditya desai <admad123(at)gmail(dot)com> |
Cc: | Pgsql Performance <pgsql-performance(at)lists(dot)postgresql(dot)org> |
Subject: | Re: Any way to speed up INSERT INTO |
Date: | 2022-03-04 18:52:28 |
Message-ID: | 0F7EBD27-C826-4172-ACA2-6B8A30BE8DEB@anarazel.de |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-performance |
Hi,
On March 4, 2022 10:42:39 AM PST, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>aditya desai <admad123(at)gmail(dot)com> writes:
>> One of the service layer app is inserting Millions of records in a table
>> but one row at a time. Although COPY is the fastest way to import a file in
>> a table. Application has a requirement of processing a row and inserting it
>> into a table. Is there any way this INSERT can be tuned by increasing
>> parameters? It is taking almost 10 hours for just 2.2 million rows in a
>> table. Table does not have any indexes or triggers.
>
>Using a prepared statement for the INSERT would help a little bit.
>What would help more, if you don't expect any insertion failures,
>is to group multiple inserts per transaction (ie put BEGIN ... COMMIT
>around each batch of 100 or 1000 or so insertions). There's not
>going to be any magic bullet that lets you get away without changing
>the app, though.
>
>It's quite possible that network round trip costs are a big chunk of your
>problem, in which case physically grouping multiple rows into each INSERT
>command (... or COPY ...) is the only way to fix it. But I'd start with
>trying to reduce the transaction commit overhead.
Pipelining could also help.
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
From | Date | Subject | |
---|---|---|---|
Next Message | Edson Richter | 2022-03-04 20:04:42 | RES: Any way to speed up INSERT INTO |
Previous Message | Bruce Momjian | 2022-03-04 18:47:35 | Re: Any way to speed up INSERT INTO |