tsvector field length limitation

From: Jonathan Marks <jonathanaverymarks(at)gmail(dot)com>
To: pgsql-general(at)postgresql(dot)org
Subject: tsvector field length limitation
Date: 2018-06-20 14:24:13
Message-ID: 5A4EDC33-60F6-4998-AA8D-9B2ED615D87C@gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Hi folks —

We utilize Postgres’ full text search system pretty heavily in our team’s operations and often index tens of millions of records with varying lengths of text. In most cases, the text we need to index is pretty short (no more than. hundreds of words) but in rare cases a single record is very very long (high hundreds of thousands of words or longer). With those records, we run into the max tsvector length requirement "The length of a tsvector (lexemes + positions) must be less than 1 megabyte”

I understand the performance implications of having very long tsvectors (our gin index updates are pretty terrible in some cases) but would really appreciate it if the max tsvector length were larger (even 5MB would make a huge difference) or if that error were a stern warning rather than a hard error.

Is there any way to disable or increase that limit in Postgres 10.3? Perhaps in a future version?

Thank you!
Jonathan

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Thomas Kellerer 2018-06-20 14:34:42 Re: Plan output: actual execution time not considering loops?
Previous Message Łukasz Jarych 2018-06-20 14:06:47 Re: Using DSN Connection and knowing windows username