Mass updates on a large table

From: "Mark Steben" <msteben(at)autorevenue(dot)com>
To: <pgsql-admin(at)postgresql(dot)org>
Subject: Mass updates on a large table
Date: 2007-08-10 17:37:07
Message-ID: 009601c7db75$12876c30$b501a8c0@D1RPQZB1
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-admin

Good afternoon,

I am attempting an update on two new date field columns on a 17 million row
table. Every row gets updated.

The update statement is a simple one:

UPDATE EMAILRCPTS SET ID = ID

And the update of the new date fields themselves occurs as the result of a
before trigger.

The update took 3 days, 10 hours to complete on the testing box. I have
already adjusted the CHECKPOINT_SEGMENTS parameter up by 3X

To minimize the impact of checkpoints. The SHARED_BUFFERS parameter has
been bumped up to 140000 on a 20meg RAM box.

There are about 9 indexes on this table although none of them reference the
date fields so since there are no inserts I don't think they would have an
impact on the update (I've been wrong before though)

Would an update statement referencing the date fields work faster than a
trigger? Do you have any other suggestions to speed this up?

We simply cannot afford this table to be down for 3+ days during a
production update. The production box is a 32meg RAM box.

We are at Postgres 7.4.5.

Thanks for your time and brainpower

Mark Steben

Senior Database Administrator
@utoRevenueT
A Dominion Enterprises Company
480 Pleasant Street
Suite B200
Lee, MA 01238
413-243-4800 Home Office
413-243-4809 Corporate Fax

msteben <blocked::mailto:name(at)autorevenue(dot)com> @autorevenue.com

Visit our new website at
<blocked::http://www.autorevenue.com/> www.autorevenue.com

IMPORTANT: The information contained in this e-mail message is confidential
and is intended only for the named addressee(s). If the reader of this
e-mail message is not the intended recipient (or the individual responsible
for the delivery of this e-mail message to the intended recipient), please
be advised that any re-use, dissemination, distribution or copying of this
e-mail message is prohibited. If you have received this e-mail message in
error, please reply to the sender that you have received this e-mail message
in error and then delete it. Thank you.

Responses

Browse pgsql-admin by date

  From Date Subject
Next Message Chris Hoover 2007-08-10 17:48:56 Re: Easy way to change table schema?
Previous Message Tom Lane 2007-08-10 16:12:49 Re: exclude logs of one database from log