Re: How to do faster DML

From: Dennis White <dwhite(at)seawardmoon(dot)com>
To: Marcos Pegoraro <marcos(at)f10(dot)com(dot)br>
Cc: Greg Sabino Mullane <htamfids(at)gmail(dot)com>, Lok P <loknath(dot)73(at)gmail(dot)com>, pgsql-general <pgsql-general(at)lists(dot)postgresql(dot)org>
Subject: Re: How to do faster DML
Date: 2024-02-04 14:44:37
Message-ID: CAE=rie-=V9Ti-3oyV4sa3+xm6tskbATnu5LsKCUEUMmOKBo6-A@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

I'm surprised no one has mentioned perhaps it's a good idea to partition
this table while adding the pk. By your own statements the table is
difficult to work with as is. Without partitioning the table, row inserts
would need to walk the pk index and could be a factor. If this is static
table then perhaps that's ok but if not...

Anyway I don't recall what type the ID was or how it's set but i suggest
you seriously investigate using it to partition this table into manageable
smaller tables.
Best of luck.

On Sun, Feb 4, 2024, 8:00 AM Marcos Pegoraro <marcos(at)f10(dot)com(dot)br> wrote:

> insert into mytable2 select * from mytable1 order by ctid limit 10_000_000
>> offset 20_000_000;
>>
>
> You can use min/max values grouping them by 10_000_000 records, so you
> don´t need that offset, then generate commands and run them.
>
> select format('insert into mytable2 select * from mytable1 where i between
> %s and %s;', max(i), min(i)) from t group by i/10_000_000;
>

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Lok P 2024-02-04 15:30:08 Re: How to do faster DML
Previous Message Marcos Pegoraro 2024-02-04 12:59:11 Re: How to do faster DML