Re: [PROPOSAL] Shared Ispell dictionaries

On Wed, Mar 07, 2018 at 10:55:29AM +0100, Tomas Vondra wrote:
> On 03/07/2018 09:55 AM, Arthur Zakirov wrote:
> > Hello Andres,
> >
> > On Thu, Mar 01, 2018 at 08:31:49PM -0800, Andres Freund wrote:
> >> Is there any chance we can instead can convert dictionaries into a form
> >> we can just mmap() into memory? That'd scale a lot higher and more
> >> dynamicallly?
> >
> > To avoid misunderstanding can you please elaborate on using mmap()? The
> > DSM approach looks like more simple and requires less code. Also DSM may
> > use mmap() if I'm not mistaken.
> >
>
> I think the mmap() idea is that you preprocess the dictionary, store the
> result in a file, and then mmap it when needed, without the expensive
> preprocessing.

Understand. I'm not againts the mmap() approach, just I have lack of
understanding mmap() benefits... Current shared Ispell approach requires
preprocessing after server restarting, and the main advantage of mmap() here
is that mmap() doesn't require preprocessing after restarting.

Speaking about the implementation.

It seems that the most appropriate place to store preprocessed files is
'pg_dynshmem' folder. File prefix could be 'ts_dict.', otherwise
dsm_cleanup_for_mmap() will remove them.

I'm not sure about reusing dsm_impl_mmap() and dsm_impl_windows(). But
maybe it's worth to reuse them.

--
Arthur Zakirov
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company