Re: [PROPOSAL] Shared Ispell dictionaries

2018-03-07 12:55 GMT+01:00 Arthur Zakirov <a(dot)zakirov(at)postgrespro(dot)ru>:

> On Wed, Mar 07, 2018 at 10:55:29AM +0100, Tomas Vondra wrote:
> > On 03/07/2018 09:55 AM, Arthur Zakirov wrote:
> > > Hello Andres,
> > >
> > > On Thu, Mar 01, 2018 at 08:31:49PM -0800, Andres Freund wrote:
> > >> Is there any chance we can instead can convert dictionaries into a
> form
> > >> we can just mmap() into memory? That'd scale a lot higher and more
> > >> dynamicallly?
> > >
> > > To avoid misunderstanding can you please elaborate on using mmap()? The
> > > DSM approach looks like more simple and requires less code. Also DSM
> may
> > > use mmap() if I'm not mistaken.
> > >
> >
> > I think the mmap() idea is that you preprocess the dictionary, store the
> > result in a file, and then mmap it when needed, without the expensive
> > preprocessing.
>
> Understand. I'm not againts the mmap() approach, just I have lack of
> understanding mmap() benefits... Current shared Ispell approach requires
> preprocessing after server restarting, and the main advantage of mmap()
> here
> is that mmap() doesn't require preprocessing after restarting.
>
> Speaking about the implementation.
>
> It seems that the most appropriate place to store preprocessed files is
> 'pg_dynshmem' folder. File prefix could be 'ts_dict.', otherwise
> dsm_cleanup_for_mmap() will remove them.
>
> I'm not sure about reusing dsm_impl_mmap() and dsm_impl_windows(). But
> maybe it's worth to reuse them.
>

I don't think so serialization to file (mmap) has not too sense. But the
shared dictionary should loaded every time, and should be released every
time if it is possible.Maybe there can be some background worker, that
holds dictionary in memory.

Regards

Pavel

>
> --
> Arthur Zakirov
> Postgres Professional: http://www.postgrespro.com
> Russian Postgres Company
>
>