Re: BUG #16647: ts_lexize with dict_voikko no longer works as of PostgreSQL 13.0

On 30/09/2020 13:19, PG Bug reporting form wrote:
> The following bug has been logged on the website:
>
> Bug reference: 16647
> Logged by: Matti Linnanvuori
> Email address: matti(dot)linnanvuori(at)portalify(dot)com
> PostgreSQL version: 13.0
> Operating system: Red Hat Enterprise Linux 8.2
> Description:
>
> ts_lexize with dict_voikko no longer works as of PostgreSQL 13.0 but it
> worked in PostgreSQL 12.4.
> dnf install postgresql13
> dnf install postgresql13-server
> dnf install postgresql13-devel
> dnf install libvoikko
> dnf install malaga-suomi-voikko
> wget
> https://www.puimula.org/htp/testing/voikko-snapshot-v5/dict-morpho.zip
> unzip dict-morpho.zip
> mkdir /etc/voikko
> mv 5 /etc/voikko
> git clone https://github.com/MattiL/dict_voikko.git
> cp -r dict_voikko /usr/pgsql-13/share/contrib
> cd /usr/pgsql-13/share/contrib/dict_voikko
> PATH="/usr/pgsql-13/bin:$PATH" make clean install USE_PGXS=1
> PGSETUP_INITDB_OPTIONS=--locale=fi_FI.utf8
> /usr/pgsql-13/bin/postgresql-13-setup initdb
> createdb -E UTF8 test
> CREATE EXTENSION dict_voikko;
> CREATE TEXT SEARCH DICTIONARY voikko_stopwords (TEMPLATE = voikko_template,
> StopWords = finnish);
> CREATE TEXT SEARCH CONFIGURATION voikko (COPY = pg_catalog.finnish);
> ALTER TEXT SEARCH CONFIGURATION voikko ALTER MAPPING FOR asciiword,
> asciihword, hword_asciipart, word, hword, hword_part WITH voikko_stopwords,
> finnish_stem;' tasks";
> test=# select ts_lexize('voikko', 'kerrostalollekohan');
> ts_lexize
> -----------
>
> (1 row)

Is that a NULL? Are you getting a NULL result for all inputs?

Did you install voikko and dict-morpho the same way on both PostgreSQL
versions?

It works for me. I installed voikko from the Debian packages with:

sudo apt install libvoikko-dev voikko-fi

And then:

git clone git(at)github(dot)com:MattiL/dict_voikko.git
cd dict_voikko
PATH=~/pgsql.13stable/bin:$PATH make USE_PGXS=1 install

And in psql:

postgres=# select version();
version

----------------------------------------------------------------------------------------
PostgreSQL 13.0 on x86_64-pc-linux-gnu, compiled by gcc (Debian
8.3.0-6) 8.3.0, 64-bit
(1 row)

postgres=# CREATE EXTENSION dict_voikko;
CREATE EXTENSION
postgres=# CREATE TEXT SEARCH DICTIONARY voikko_stopwords (TEMPLATE =
voikko_template,
StopWords = finnish);
CREATE TEXT SEARCH DICTIONARY
postgres=# CREATE TEXT SEARCH CONFIGURATION voikko (COPY = finnish);
CREATE TEXT SEARCH CONFIGURATION
postgres=# ALTER TEXT SEARCH CONFIGURATION voikko ALTER MAPPING FOR
asciiword,
asciihword, hword_asciipart, word, hword, hword_part WITH voikko_stopwords,
finnish_stem;
ALTER TEXT SEARCH CONFIGURATION
postgres=# select ts_lexize('voikko', 'kerrostalollekohan');
ts_lexize
---------------
{kerros,talo}
(1 row)

- Heikki