Re: Degraded performance during table rewrite

Mohamed Wael Khobalatte <mkhobalatte(at)grubhub(dot)com> writes:
> Tom, I think I understood what you meant after getting some sleep. What I
> do now is:

> - Initially load a lot of ids, say a million.
> - Then I slice that array by my preferred batch, say a 10000.
> - Loop through these slices and run an improved query, which looks like
> this:

> WITH del AS (
> DELETE FROM #{old_table}
> WHERE id >= #{first_id_in_slice} AND id <= #{last_id_in_slice}
> RETURNING *
> )
> INSERT INTO #{table}
> SELECT * FROM del
> RETURNING id

Right, the problem occurs when you're deleting stuff that's at the end
of the id range, and have already deleted stuff that was previously the
end of the range. If you start somewhere in the middle and work towards
the end, you dodge that problem, even for the very last delete that
*is* handling the endmost ids.

regards, tom lane