Re: [GENERAL] Creation of tsearch2 index is very slow

From: Oleg Bartunov <oleg(at)sai(dot)msu(dot)su>
To: Martijn van Oosterhout <kleptog(at)svana(dot)org>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Ron <rjpeace(at)earthlink(dot)net>, pgsql-performance(at)postgresql(dot)org
Subject: Re: [GENERAL] Creation of tsearch2 index is very slow
Date: 2006-01-21 15:24:30
Message-ID: Pine.GSO.4.63.0601211824060.14417@ra.sai.msu.su
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-performance

gevel is available from
http://www.sai.msu.su/~megera/postgres/gist/

Oleg
On Sat, 21 Jan 2006, Martijn van Oosterhout wrote:

> On Sat, Jan 21, 2006 at 04:29:13PM +0300, Oleg Bartunov wrote:
>> Martijn, you're right! We want not only to split page to very
>> different parts, but not to increase the number of sets bits in
>> resulted signatures, which are union (OR'ed) of all signatures
>> in part. We need not only fast index creation (thanks, Tom !),
>> but a better index. Some information is available here
>> http://www.sai.msu.su/~megera/oddmuse/index.cgi/Tsearch_V2_internals
>> There are should be more detailed document, but I don't remember where:)
>
> I see how it works, what I don't quite get is whether the "inverted
> index" you refer to is what we're working with here, or just what's in
> tsearchd?
>
>>> That's harder though (this algorithm does approximate it sort of)
>>> and I havn't come up with an algorithm yet
>>
>> Don't ask how hard we thought :)
>
> Well, looking at how other people are struggling with it, it's
> definitly a Hard Problem. One thing though, I don't think the picksplit
> algorithm as is really requires you to strictly have the longest
> distance, just something reasonably long. So I think the alternate
> algorithm I posted should produce equivalent results. No idea how to
> test it though...
>
> Have a nice day,
>

Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru)
Sternberg Astronomical Institute, Moscow University, Russia
Internet: oleg(at)sai(dot)msu(dot)su, http://www.sai.msu.su/~megera/
phone: +007(495)939-16-83, +007(495)939-23-83

In response to

Browse pgsql-general by date

  From Date Subject
Next Message Ron 2006-01-21 16:11:53 Re: [GENERAL] Creation of tsearch2 index is very
Previous Message Oleg Bartunov 2006-01-21 15:22:52 Re: [GENERAL] Creation of tsearch2 index is very slow

Browse pgsql-performance by date

  From Date Subject
Next Message Ron 2006-01-21 16:11:53 Re: [GENERAL] Creation of tsearch2 index is very
Previous Message Oleg Bartunov 2006-01-21 15:22:52 Re: [GENERAL] Creation of tsearch2 index is very slow