Re: string_to_array eats too much memory?

>> tsvector size should not be greater than 1Mb however.
>
> Is this documented somewhere? Also I noticed that tsearch2 treats ":"
> as a special character. Are there any special characters? If so where
> are they documented?
http://www.sai.msu.su/~megera/wiki/Tsearch_V2_in_Brief
Limitations

13.1 2048 bytes for lexems
13.2 ts_vector has limit about 1Mb. Exact value depends on
quantity of position information. If there is no any position
information, then sum of length of lexem must be less than 1Mb,
otherwise, sum of length of and pos. info.
Positional information uses 2 bytes per each
position and 2 bytes per lexem with pos info. The number of
lexems is limited by 4^32, so in practice it's unlimited.
13.3 ts_query:
Number of entries (nodes, i.e sum of lexems and operation)
is limited: internal representation is in polish notation
and position of one operand is pointed by int2, so it's
rather soft limit.
In any case, low range of limit - 32768 nodes.
Notice: ts_query doesn't designed for storing in table and
is optimized for speed, not for size.
13.4 Positional information in ts_vector:
13.4.1 Value of position may not be greater than 2^14 (16384),
any value greater than this limit will be replaced
by 16383.
13.4.2 Only 256 positional info per lexem.

Some useful articles
http://www.sai.msu.su/~megera/postgres/gist/tsearch/V2/docs/HOWTO-parser-tsearch2.html
http://www.sai.msu.su/~megera/postgres/gist/tsearch/V2/docs/custom-dict.html

--
Teodor Sigaev E-mail: teodor(at)sigaev(dot)ru
WWW: http://www.sigaev.ru/