Re: GIN fast insert

On Mon, Feb 23, 2009 at 10:05 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Robert Haas <robertmhaas(at)gmail(dot)com> writes:
>> I'm starting to think that the right thing to do here is to create a
>> non-lossy option for TIDBitmap. Tom has been advocating just losing
>> the index scan AM altogether, but that risks losing performance in
>> cases where a LIMIT would have stopped the scan well prior to
>> completion.
>
> Actually, I'm going to *insist* that we lose the index AM scan
> altogether. There might be a possibility to put it back in 8.5 or later
> if anyone actually makes the case (with some evidence) that it's worth
> the trouble. But right now, what this patch needs is to be made to work
> reliably, and having a great deal of complexity added by an inessential
> feature is a good way to make sure it doesn't go in at all.

Except that the "inessential" feature in question is a feature that
currently WORKS, and I don't believe that the testing you've done is
anywhere near sufficient to show that no one will be upset if it goes
away. Without some convincing evidence to support that proposition, I
think it would be better to postpone the whole patch to 8.5 and use
that time to fix the problem, rather than slam something through now
and then possibly find out that we've introduced a performance
regression in cases that used to work well.

In fact, as far as I can see, the performance testing that has been
done on this patch to date is fairly one-sided. It focuses on the
improvement to index build speed, which is a good thing to test, but I
don't believe anyone has done any testing at all of the cases where
the patch might HURT. Previously, the relevant case was where someone
has done a bunch of fast-inserts, but autovacuum has not kicked in
yet, or the threshold to trigger autovacuum has not been reached. Now
we're adding to that every query that currently uses an index scan,
which seemed to be a pretty common plan in the simple cases I tested.

Maybe I'm worrying over nothing?

...Robert