Re: Batch update of indexes on data loading

On Tue, 2008-02-26 at 09:08 +0000, Simon Riggs wrote:

> I very much like the idea of index merging, or put another way: batch
> index inserts. How big do the batch of index inserts have to be for us
> to gain benefit from this technique? Would it be possible to just buffer
> the index inserts inside the indexam module so that we perform a batch
> of index inserts every N rows? Maybe use work_mem? Or specify a batch
> size as a parameter on COPY?

Itagaki,

I think the index merging idea is still useful even if we do not build a
whole new index. ISTM we can do this without locking the table also.

I understand it is most efficient when you do rebuild the index by
merging the old index with the incoming data, but it does seem there are
other problems associated with doing that.

Your idea still has a great future, IMHO.

--
Simon Riggs
2ndQuadrant http://www.2ndQuadrant.com