Re: Can tsearch do some basic text mining

From: Oleg Bartunov <oleg(at)sai(dot)msu(dot)su>
To: Phoenix Kiula <phoenix(dot)kiula(at)gmail(dot)com>
Cc: Postgres General <pgsql-general(at)postgresql(dot)org>
Subject: Re: Can tsearch do some basic text mining
Date: 2007-08-24 17:53:23
Message-ID: Pine.LNX.4.64.0708242151450.2727@sn.sai.msu.ru
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

On Fri, 24 Aug 2007, Phoenix Kiula wrote:

> Hi,
>
> We have big blobs of text (average 10,000 characters) in a database,
> from which we would like to discover the most often repeated words or
> phrases. Can tsearch be used for this kind of pattern search? I
> suppose it's Text Mining 101 sort of stuff, nothing complex.

there is stat() function, see
http://www.sai.msu.su/~megera/wiki/Tsearch_V2_Notes
for more details.
It's not fast, so better to save results in a table

>
> TIA!
>
> ---------------------------(end of broadcast)---------------------------
> TIP 2: Don't 'kill -9' the postmaster
>

Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru)
Sternberg Astronomical Institute, Moscow University, Russia
Internet: oleg(at)sai(dot)msu(dot)su, http://www.sai.msu.su/~megera/
phone: +007(495)939-16-83, +007(495)939-23-83

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Joshua D. Drake 2007-08-24 18:13:52 Re: Out of Memory - 8.2.4
Previous Message Tom Lane 2007-08-24 17:50:37 Re: FATAL: could not reattach to shared memory (Win32)