Re: snowball ASCII stemmer configuration

On 2020-06-16 16:37, Tom Lane wrote:
> After further reflection, I think these are indeed mistakes and we should
> change them all. The argument for the Russian/English case, AIUI, is
> "if we come across an all-ASCII word, it is most certainly not Russian,
> and the most likely Latin-based language is English". Given the world
> as it is, I think the same argument works for all non-Latin-alphabet
> languages. Obviously specific applications might have a different idea
> of the best fallback language, but that's why we let users make their
> own text search configurations. For general-purpose use, falling back
> to English seems reasonable. And we can be dead certain that applying
> a Greek stemmer to an ASCII word will do nothing useful, so the
> configuration choice shown above is unhelpful.

Do we *have* to have an ASCII stemmer that corresponds to an actual
language? Couldn't we use the simple stemmer or no stemmer at all?

In my experience, ASCII text in, say, Russian or Greek will typically be
acronyms or brand names or the like, and there doesn't seem to be a
great need to stem that kind of thing. Just doing nothing seems at
least as good.

--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services