From: | Jan Urbański <wulczer(at)wulczer(dot)org> |
---|---|
To: | PostgreSQL - Hans-Jürgen Schönig <postgres(at)cybertec(dot)at> |
Cc: | pgsql-hackers Hackers <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: Caching Python modules |
Date: | 2011-08-17 12:20:55 |
Message-ID: | 4E4BB227.1000108@wulczer.org |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On 17/08/11 14:19, Jan Urbański wrote:
> On 17/08/11 14:09, PostgreSQL - Hans-Jürgen Schönig wrote:
>> CREATE OR REPLACE FUNCTION textprocess.add_to_corpus(lang text, t text) RETURNS float4 AS $$
>>
>> from SecondCorpus import SecondCorpus
>> from SecondDocument import SecondDocument
>>
>> i am doing some intense text mining here.
>> the problem is: is it possible to cache those imported modules from function to function call.
>> GD works nicely for variables but can this actually be done with imported modules as well?
>> the import takes around 95% of the total time so it is definitely something which should go away somehow.
>> i have checked the docs but i am not more clever now.
>
> After a module is imported in a backend, it stays in the interpreter's
> sys.modules dictionary and importing it again will not cause the module
> Python code to be executed.
>
> As long as you are using the same backend you should be able to call
> add_to_corpus repeatedly and the import statements should take a long
> time only the first time you call them.
>
> This simple test demonstrates it:
>
> [example missing the slow() function code]
Oops, forgot to show the CREATE statement of the test function:
postgres=# create or replace function slow() returns void language
plpythonu as $$ import slow $$;
Jan
From | Date | Subject | |
---|---|---|---|
Next Message | Robert Haas | 2011-08-17 12:37:06 | Re: Backup's from standby |
Previous Message | Jan Urbański | 2011-08-17 12:19:02 | Re: Caching Python modules |