| From: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
|---|---|
| To: | Tim <elatllat(at)gmail(dot)com> |
| Cc: | pgsql-admin(at)postgresql(dot)org |
| Subject: | Re: tsvector limitations |
| Date: | 2011-06-16 03:14:06 |
| Message-ID: | 14751.1308194046@sss.pgh.pa.us |
| Views: | Raw Message | Whole Thread | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-admin |
Tim <elatllat(at)gmail(dot)com> writes:
> My limited testing indicates tsvector size has an approximately linear
> (binomial) correlation with "number of unique words", and "word count".
Yes, it would be linear in the number of unique words (assuming the
average word length stays constant). It's not linear in word count,
because tsvector only keeps track of distinct word positions for the
first 16K words in the text. So for large documents only the number of
unique words is really going to matter, and that grows *much* less than
linearly with document size for real-world documents.
regards, tom lane
| From | Date | Subject | |
|---|---|---|---|
| Next Message | bakkiya | 2011-06-16 05:28:06 | could not open temporary statistics file "pg_stat_tmp/pgstat.tmp" |
| Previous Message | Tim | 2011-06-16 02:34:01 | Re: tsvector limitations |