Re: Combine Top-k with similarity search extensions

From: Shmagi Kavtaradze <kavtaradze(dot)s(at)gmail(dot)com>
To: tim(dot)child(at)comcast(dot)net
Cc: pgsql-novice(at)postgresql(dot)org
Subject: Re: Combine Top-k with similarity search extensions
Date: 2015-11-20 16:13:15
Message-ID: CAHY6mawC2GN=W7gRDWgDhv9CTLLTmpU=Wq9jBAsTKtjPCMTEOg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-novice

It will add complexity and also no idea how to do it. Is there any
alternative?

On Fri, Nov 20, 2015 at 5:00 PM, <tim(dot)child(at)comcast(dot)net> wrote:

> Shmagi,
>
> Take the first 20 text characters and compute and store the CRC32 or MD5
> of that value. That value acts as a signature. You can then find all
> distinct signatures, or all rows with duplicate signatures for further
> analysis You could event try building a signature on the full text string.
>
>
>
> ------------------------------
> *From: *"Shmagi Kavtaradze" <kavtaradze(dot)s(at)gmail(dot)com>
> *To: *pgsql-novice(at)postgresql(dot)org
> *Sent: *Friday, November 20, 2015 2:21:36 AM
> *Subject: *[NOVICE] Combine Top-k with similarity search extensions
>
> I am performing similarity check over a column in a table with about 3500
> entries. Column is populated with text data from text file. Performing a
> check results in 3500 * 3500 rows and it takes forever to calculate for my
> virtual machine. Is there any way to calculate for top-k results, to
> decrease amount and time needed? What I mean is that, for example when
> checking two sentences, if first several words does not match, to stop
> checking that sentences and move on.
>
>

In response to

Responses

Browse pgsql-novice by date

  From Date Subject
Next Message tim.child 2015-11-20 16:42:44 Re: Combine Top-k with similarity search extensions
Previous Message tim.child 2015-11-20 16:00:30 Re: Combine Top-k with similarity search extensions