Re: tsearch profiling - czech environment - take 55MB

2010/3/11 Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>:
> Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com> writes:
>> There are some wrong in our implementation NISortDictionary. After
>> initialisation is ts_cache memory context 55MB long and pg takes
>> 190MB.
>
> What's your tsearch configuration exactly?
>

files: http://www.pgsql.cz/data/czech.tar.gz

configuration:

CREATE TEXT SEARCH DICTIONARY cspell
(template=ispell, dictfile = czech, afffile=czech, stopwords=czech);
CREATE TEXT SEARCH CONFIGURATION cs (copy=english);
ALTER TEXT SEARCH CONFIGURATION cs
ALTER MAPPING FOR word, asciiword WITH cspell, simple;

then try: select * from ts_debug('cs','Příliš žluťoučký kůň se napil
žluté vody');

with some time (used fce clock())

cspell: 1024 total in 1 blocks; 136 free (1 chunks); 888 used
After dictionary loading 320000
cspell: 3072 total in 2 blocks; 568 free (5 chunks); 2504 used
Ispell dictionary init context: 27615288 total in 13 blocks; 7710864
free (12 chunks); 19904424 used
After AffFile loading 330000
cspell: 816952 total in 78 blocks; 18072 free (18 chunks); 798880 used
Ispell dictionary init context: 27615288 total in 13 blocks; 7710864
free (20 chunks); 19904424 used
After stop words loading 330000
cspell: 816952 total in 78 blocks; 13360 free (13 chunks); 803592 used
Ispell dictionary init context: 27615288 total in 13 blocks; 7710864
free (20 chunks); 19904424 used
****** 1 ******
cspell: 816952 total in 78 blocks; 9240 free (12 chunks); 807712 used
Ispell dictionary init context: 27615288 total in 13 blocks; 7710864
free (20 chunks); 19904424 used
****** 2 ****** 380000
cspell: 825144 total in 79 blocks; 8440 free (10 chunks); 816704 used
Ispell dictionary init context: 27615288 total in 13 blocks; 7710864
free (20 chunks); 19904424 used
****** 2.5 ****** 490000
// mkSPNode
cspell: 825144 total in 79 blocks; 8440 free (10 chunks); 816704 used
Ispell dictionary init context: 27615288 total in 13 blocks; 7710864
free (20 chunks); 19904424 used
****** 3 ****** 580000
cspell: 55706480 total in 6775 blocks; 140200 free (1728 chunks); 55566280 used
Ispell dictionary init context: 27615288 total in 13 blocks; 7710864
free (20 chunks); 19904424 used
After dictionary sort 580000
cspell: 55706480 total in 6775 blocks; 140200 free (1728 chunks); 55566280 used
Ispell dictionary init context: 27615288 total in 13 blocks; 7710864
free (20 chunks); 19904424 used
After Affixes sort 580000
cspell: 55853736 total in 6789 blocks; 130208 free (1553 chunks); 55723528 used
Ispell dictionary init context: 27615288 total in 13 blocks; 7710864
free (34 chunks); 19904424 used
final 580000
cspell: 55853736 total in 6789 blocks; 130208 free (1553 chunks); 55723528 used
Ispell dictionary init context: 27615288 total in 13 blocks; 7710864
free (34 chunks); 19904424 used
executor start

>                        regards, tom lane
>