Re: Any way to speed up INSERT INTO

From: Imre Samu <pella(dot)samu(at)gmail(dot)com>
To: aditya desai <admad123(at)gmail(dot)com>
Cc: Pgsql Performance <pgsql-performance(at)lists(dot)postgresql(dot)org>
Subject: Re: Any way to speed up INSERT INTO
Date: 2022-03-05 01:22:13
Message-ID: CAJnEWwmRbA89SjyNx8hcuP0upTtrMG+pi3aOu9=JkMpYp6gvCw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-performance

> Correct rows are wider. One of the columns is text and one is bytea.

with the PG14 the LZ4 compression is worth checking.

via
https://www.postgresql.fastware.com/blog/what-is-the-new-lz4-toast-compression-in-postgresql-14

*"""INSERT statements with 16 clientsAnother common scenario that I tested
was accessing the database from multiple clients - 16 in this case.What I
found out, as can be seen below, is that compression performance of single
large files (HTML, English text, source code, executable binary, pictures)
using LZ4 was 60% to 70% faster compared to PGLZ, and that there was also a
small improvement while inserting multiple small files (PostgreSQL
document).*
*"""*

kind regards,
Imre

aditya desai <admad123(at)gmail(dot)com> ezt írta (időpont: 2022. márc. 4., P,
19:42):

> Hi Bruce,
> Correct rows are wider. One of the columns is text and one is bytea.
>
> Regards,
> Aditya.
>
> On Sat, Mar 5, 2022 at 12:08 AM Bruce Momjian <bruce(at)momjian(dot)us> wrote:
>
>> On Sat, Mar 5, 2022 at 12:01:52AM +0530, aditya desai wrote:
>> > Hi,
>> > One of the service layer app is inserting Millions of records in a
>> table but
>> > one row at a time. Although COPY is the fastest way to import a file in
>> a
>> > table. Application has a requirement of processing a row and inserting
>> it into
>> > a table. Is there any way this INSERT can be tuned by increasing
>> parameters? It
>> > is taking almost 10 hours for just 2.2 million rows in a table. Table
>> does not
>> > have any indexes or triggers.
>>
>> Well, sections 14.4 and 14.5 might help:
>>
>> https://www.postgresql.org/docs/14/performance-tips.html
>>
>> Your time seems very slow --- are the rows very wide?
>>
>> --
>> Bruce Momjian <bruce(at)momjian(dot)us> https://momjian.us
>> EDB https://enterprisedb.com
>>
>> If only the physical world exists, free will is an illusion.
>>
>>

In response to

Browse pgsql-performance by date

  From Date Subject
Next Message Mladen Gogala 2022-03-05 02:33:01 XA transactions much slower on 14.2 than on 13.5
Previous Message Edson Richter 2022-03-04 20:04:42 RES: Any way to speed up INSERT INTO