Re: python modul pre-import to avoid importing each time

Hey,
thanks for your answer !

Yep you are right, the function I would like to test are going to be called
a lot (100k times), so even 15 ms per call matters.

I'm still a bit confused by a topic I found here :
http://stackoverflow.com/questions/15023080/how-are-import-statements-in-plpython-handled
The answer gives a trick to avoid importing each time, so somehow it must
be usefull.

On another internet page (can't find it anymore) somebody mentioned this
module loading at server startup, one way or another, but gave no
precision. It seems that the "plpy" python module get loaded by default,
would'nt it be possible to hack this module to add other import inside it?

I also use PL/R (untrusted I guess) and you can create a special table to
indicate which module to load at startup.

Cheers,
Rémi-C

2014-06-25 21:46 GMT+02:00 Jeff Janes <jeff(dot)janes(at)gmail(dot)com>:

> On Thu, Jun 19, 2014 at 7:50 AM, Rémi Cura <remi(dot)cura(at)gmail(dot)com> wrote:
> > Hey List,
> >
> > I use plpython with postgis and 2 python modules (numpy and shapely).
> > Sadly importing such module in the plpython function is very slow
> (several
> > hundreds of milliseconds).
>
> Is that mostly shapely (which I don't have)? numpy seems to be pretty
> fast, like 16ms. But that is still slow for what you want, perhaps.
>
> >
> > I also don't know if this overhead is applied each time the function is
> > called in the same session.
>
> It is not. The overhead is once per connection, not once per call.
> So using a connection pooler could be really be a help here.
>
> > Is there a way to pre-import those modules once and for all,
> > such that the python function are accelerated?
>
> I don't think there is. With plperl you can do this by loading the
> module in plperl.on_init and by putting plperl into
> shared_preload_libraries so that this happens just at server start up.
> But I don't see a way to do something analogous for plpython due to
> lack of plpython.on_init. I think that is because the infrastructure
> to do that is part of making a "trusted" version of the language,
> which python doesn't have. (But it could just be that no one has ever
> gotten around to adding it.)
>
> Cheers,
>
> Jeff
>