Re: Possible major bug in PlPython (plus some other ideas)

Hi Bradley,

Thanks for the response! I'm very relieved to get feedback on my
suggestions.

On Fri, 9 Nov 2001, Bradley McLean wrote:

> * Kevin Jacobs (jacobs(at)penguin(dot)theopalgroup(dot)com) [011109 08:59]:
> >
> > 1) If Plpython is installed as a trusted language, and from what little I
> > can glean from the documentation, it should not have any filesystem access.
> > However, the default behavior of the restricted execution environment
> > being used allows read-only filesystem access.
> >...
> > It is fairly easy to override this method to unconditionally raise an
> > access exception.
>
> Agreed. Just to amplify the point below, there is currently no
> distinction between trusted and untrusted in PLpython (hmm, need
> a doc change to identify this?). lanpltrusted appears nowhere
> in the implementation.

Actually, I'm mostly unaware of how createlang works, but I assumed that
plpython was installed as a trusted backend. Can anyone clarify _exactly_
what is meant by a 'trusted'?

To fix this issue, is it better to create a C-API method override to RExec
or should we implement a real plpy.py module that does this (and some of the
rest of my suggestions). I'd prefer the module, since it is _much_ easier
to do, though it does incur some extra overhead.

> > 4) Plpython does not use the standard Python boolean checks, which is also
> > not very Pythonic and somewhat confusing:
> > ...
> > I suggest changing the semantics of boolean return values to use
> > PyObject_IsTrue(PyObject *o) to properly test for truth values.
>
> Agree. Is this the only type that needs special treatment?

Virtually all the other basic types will usually do the right thing.
We could add support for array conversions:

Python [1,2,3] => Postgres INTEGER[] {1,2,3}

Can you think of any others?

> > 6) [Very low priority] Its not insane to consider a Plpython procedure
> > that spawns off a Python thread to do background processing tasks.
> > This is obviously something that will only be possible in an untrusted
> > version of the interpreter. Also, if the SPI interface is thread-safe,
> > then it may be useful to release the Python interpreter lock around
> > some of the SPI calls.
>
> Three weeks ago, I really wanted a feature like this, but I've slowly
> been convincing myself that it's a bad idea. Consider the effects of
> a race condition in user generated threaded python code that results
> in a deadlock within the backend.

I'm not completely sold on this either. I'm going to let this idea
percolate for a while and wait to see if I can find a need for it.

> I will say that the discrepancy between executing SQL from plpython
> and from python with the pg driver does cause some cognitive
> disconnect, because despite working in the same language, I have
> to use different APIs to the same functionality. I don't have a
> good proposal to solve it, though.

Other than not having the same modules available, what is annoying you most?

> One last issue:
>
> 7) Would it be completely impossible to make an extra-global
> dictionary that was shared between multiple back-ends (place
> in shared memory)? Anything I'm likely to place in GD, I'm likely
> to initialize once and want in all backends.

It will be very tricky to do in full generality. Anything that is inserted
into the "shared global" dictionary will have to be copied, re-allocated on
the shared heap, and hidden from the garbage collector. This is fairly easy
to do for a simple types {int,float,string}; much harder for more complex
types.

-Kevin

--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19 E-mail: jacobs(at)theopalgroup(dot)com
Fax: (216) 986-0714 WWW: http://www.theopalgroup.com