how does full text searching tokenize words ? can it be altered?

From: Jonathan Vanasco <postgres(at)2xlp(dot)com>
To: PostgreSQL general <pgsql-general(at)postgresql(dot)org>
Subject: how does full text searching tokenize words ? can it be altered?
Date: 2014-07-11 00:02:44
Message-ID: 93848FE9-AECD-4044-AB1E-AD612FCDB09A@2xlp.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general


I'm getting a handful of 'can not index words longer than 2047 characters' on my `gin` indexes.

1. does this 2047 character count correspond to tokens / indexed words?
2. if so, is there a way to lower this number ?
3. is there a way to profile the index for the frequency of tokens ?

( apologies in advance if this looks familiar, i posted this as part of a larger question last month; everything but this was answered by the list and I can't find answers to this online )

Browse pgsql-general by date

  From Date Subject
Next Message Steve Atkins 2014-07-11 00:26:44 Re: Windows Installation User account - Correct database for us
Previous Message Don Brown 2014-07-11 00:01:38 Windows Installation User account - Correct database for us