Re: 8.4 open item: copy performance regression?

From: Greg Stark <gsstark(at)mit(dot)edu>
To: Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Stefan Kaltenbrunner <stefan(at)kaltenbrunner(dot)cc>, Simon Riggs <simon(at)2ndquadrant(dot)com>, Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com>, Greg Smith <gsmith(at)gregsmith(dot)com>, Andrew Dunstan <andrew(at)dunslane(dot)net>, Kevin Grittner <Kevin(dot)Grittner(at)wicourts(dot)gov>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>, Alan Li <ali(at)truviso(dot)com>
Subject: Re: 8.4 open item: copy performance regression?
Date: 2009-06-21 16:15:14
Message-ID: 407d949e0906210915i3c8e2682kf8d8939ee38efd2d@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Sun, Jun 21, 2009 at 5:07 PM, Robert Haas<robertmhaas(at)gmail(dot)com> wrote:
>
> I think we had the idea of buffering up enough tuples to fill a page
> (estimating conservatively so as to make sure we actually fill it),
> and then inserting them all at once.  But I'm not sure how much
> trouble that causes in terms of the timing of inserting index entries
> and firing of after row insert triggers.  If the command ID doesn't
> change, it seems like it might be OK.  Or at worst, even if the
> optimization would only work in cases where there are no after row
> triggers, that would still be useful to a lot of people, I think.
>
> I haven't really spent much time on this angle of attack and
> completely confess to not understanding all of the issues...

There was some discussion of doing this in general for all inserts
inside the indexam. The btree indexam could buffer up any inserts done
within the transaction and keep them in an in-memory btree. Any
lookups done within the transaction first look up in the in-memory
tree then the disk. If the in-memory buffer fills up then we flush
them to the index.

The reason this is tempting is that we could then insert them all in a
single index-merge operation which would often be more efficient than
retail inserts.

--
greg
http://mit.edu/~gsstark/resume.pdf

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2009-06-21 16:38:34 Re: 8.4 open item: copy performance regression?
Previous Message Robert Haas 2009-06-21 16:11:36 Re: 8.4 open item: copy performance regression?