Re: Text Search vs MYSQL vs Lucene

On Thu, Sep 09, 2004 at 07:56:20AM -0500, Vic Cekvenich wrote:

> What would be performance of pgSQL text search vs MySQL vs Lucene (flat
> file) for a 2 terabyte db?
> thanks for any comments.

My experience with tsearch2 has been that indexing even moderately
large chunks of data is too slow to be feasible. Moderately large
meaning tens of megabytes.

Your milage might well vary, but I wouldn't rely on postgresql full
text search of that much data being functional, let alone fast enough
to be useful. Test before making any decisions.

If it's a static or moderately static text corpus you're probably
better using a traditional FTS system anyway (tsearch2 has two
advantages - tight integration with pgsql and good support for
incremental indexing).

Two terabytes is a lot of data. I'd suggest you do some research on
FTS algorithms rather than just picking one of the off-the-shelf FTS
systems without understanding what they actually do. "Managing
Gigabytes" ISBN 1-55860-570-3 covers some approaches.

Cheers,
Steve