Re: trying to delete most of the table by range of date col

From: Mariel Cherkassky <mariel(dot)cherkassky(at)gmail(dot)com>
To: Andreas Kretschmer <andreas(at)a-kretschmer(dot)de>
Cc: pgsql-performance(at)lists(dot)postgresql(dot)org
Subject: Re: trying to delete most of the table by range of date col
Date: 2018-09-03 08:17:58
Message-ID: CA+t6e1=VV2M8Fqm0txKrof4s-OscLzuGu7Ekq5HnNSZL-GKM_w@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-admin pgsql-performance

Hi,
I already checked and on all the tables that uses the id col of the main
table as a foreign key have index on that column.
I tried all the next 4 solutions :

1)delete from my_table where end_date <=
to_date('12/12/2018','DD/MM/YYYY') and end_date >
to_date('11/12/2018','DD/MM/YYYY');
Execution time: 350603.047 ms ~ 5.8 minutes

2)DELETE FROM my_table WHERE id IN (select id from my_table where end_date
<= to_date('12/12/2018','DD/MM/YYYY') and end_date >
to_date('11/12/2018','DD/MM/YYYY'));
Execution time: 333166.233 ms ~ 5.5 minutes

3) set temp_buffers='1GB';
SET

create temp table id_temp as select id from my_Table where end_date <=
to_date('12/12/2018','DD/MM/YYYY') and end_date >
to_date('11/12/2018','DD/MM/YYYY') ;
SELECT 1572864
Time: 2196.670 ms

DELETE FROM my_table USING id_temp WHERE my_table.id = id_temp.id;
Execution time: 459650.621 ms 7.6minutes

4)delete in chunks :
do $$
declare
rec integer;
begin
select count(*) from my_table into rec where end_date <=
to_date('12/12/2018','DD/MM/YYYY') and end_date >
to_date('11/12/2018','DD/MM/YYYY');
while rec > 0 loop
DELETE FROM my_Table WHERE id IN (select id from my_tablewhere end_date <=
to_date('12/12/2018','DD/MM/YYYY') and end_date >
to_date('11/12/2018','DD/MM/YYYY') limit 5000);
rec := rec - 5000;
raise notice '5000 records were deleted, current rows :%',rec;
end loop;

end;
$$
;

Execution time : 6 minutes.

So, it seems that the second solution is the fastest one. It there a reason
why the delete chunks (solution 4) wasnt faster?

‫בתאריך יום ב׳, 3 בספט׳ 2018 ב-10:35 מאת ‪Andreas Kretschmer‬‏ <‪
andreas(at)a-kretschmer(dot)de‬‏>:‬

>
>
> Am 03.09.2018 um 09:06 schrieb Justin Pryzby:
> > Note, I believe it's planned in the future for foreign keys to support
> > referenes to partitioned tables, at which point you could just DROP the
> monthly
> > partition...but not supported right now.
>
> the future is close, that's possible in 11 ;-)
>
> Regards, Andreas
>
> --
> 2ndQuadrant - The PostgreSQL Support Company.
> www.2ndQuadrant.com
>
>
>

In response to

Responses

Browse pgsql-admin by date

  From Date Subject
Next Message pavan95 2018-09-03 08:26:11 Re: Heavy Logging in Subscriber side when configured Logical Replication in 10.4
Previous Message Achilleas Mantzios 2018-09-03 08:04:04 Re: Heavy Logging in Subscriber side when configured Logical Replication in 10.4

Browse pgsql-performance by date

  From Date Subject
Next Message Sergei Kornilov 2018-09-03 08:35:05 Re: trying to delete most of the table by range of date col
Previous Message Andreas Kretschmer 2018-09-03 07:35:16 Re: trying to delete most of the table by range of date col