[Pljava-dev] The pljava Project is dead (LONG but I hope interesting)

On 12/02/15 18:08, Jos? Carlos Mart?nez Llario wrote:
> Im very sorry to hear in other posts from potsgres community that
> pljava is almost dead.
> Is this true? hope not but im feeling like is already dead.
> Hope someone says the opposite.

I certainly hope it is not dead ... it's working too well for me, and
I've only recently regained enough time to contribute to it some more
myself.

Maybe it'll help to give some of my impressions of the condition of
the project, as a relative newcomer who wasn't around for the early
development, but successfully uses it and contributes to it.

MY CURRENT IMPRESSIONS:

I started with PL/Java just before the switch to maven-based building in
2013. I found the old build a bit fiddly, and then I grumbled a little
about having to learn this maven thing, but I have to say now that the
maven setup easily builds me a working pljava every time and seems quite
solid. Currently my platform is PG9.3 and Linux ... maybe other
platforms have build issues that could be improved.

Also, once built and installed in postgres, pljava seems to Just Work
and do what it says on the tin. For a new user, the process of getting
it set up in postgres, the first time, is more fiddly than it could be,
and that's where some valuable improvements could be made.

So, from where I stand, here's what I see:

* It's usable now, for many realistic purposes

* It has areas where valuable improvement could still be made
(especially "around the edges", simplifying first-time installation,
etc.)

* The improvable bits tend to be small/meaty enough for one person, or
just a few, to tackle and add noticeable value; that's kind of nice.

Thomas Hallgren seems to not currently have a lot of time to pound
on it himself, but he is clearly still paying attention, and responds
promptly when I make pull requests, usually in under a day. I am not
sure how many other people have commit access.

THINGS HOLDING IT BACK:

Now, for the things I see holding it back, or being discouraging
to newcomers. You'll see how many I think could be easily addressed:

* The wiki is way overdue for some love. For example, the maven-based
build is working great as far as I can tell, and the makefiles are
obsolete, but the wiki 'building' page doesn't even say maven yet
(issue 18). I've been making code pull requests lately, but I can
try to fit in some wiki attention too, if I can get edit permissions.

* The javadocs aren't viewable on the github site. There is plenty of
good javadoc in the code, and I've just messed around a little with
the site-generating plugin for maven, and like magic produced a web
site image with the javadocs included. I think it would be great to
push that to the project's github-pages space, which is currently
unused. (I can see that Thomas was experimenting with it in early
2013, but that was just the same content that's now on the wiki.
I think it could be quite helpful to keep the user-oriented texts
on the wiki where they are now, but allow them to link to the
detailed technical docs on github-pages.)

* The issue tracker doesn't have a huge number of issues, but some are
rather old. Also they are a mix of true distinct issues, and some
duplicates or relateds, and some that should just have been list
questions. Even just a small increase in attention to reviewing/
categorizing/triaging issues when filed would probably go a long
way to making the issue tracker seem less stagnant. I've been doing
my part this week to try and resolve a few of 'em.

* Issue 11, about regression testing, is super important, because a
lack of good regression testing can have a chilling effect on
participation. It is scary to dive in and start banging on code when
the tests are sparse and fiddly to run. I agree with Thomas that it
would certainly be a start to take the existing "examples" and
scaffold them as tests, and I think I've seen an easy way to do it,
that I'll work on. Once there's better testing, the project can be
much more inviting to work on, because the fear-of-hosing-something
factor is less.

TIME TO REVISIT BACKWARD COMPATIBILITY TARGETS ?

I think it would help for the project to publish an updated cutoff
for the oldest versions of PostgreSQL and Java that it actively means
to support. The latest I can find are in the old readme.html still
viewable in docs/ that says PostgreSQL >= 8.0.3 and Java >= 1.4.
According to 'git blame', both those lines were last touched
in 2005. :) The intro.html in the same directory says "primarily
targeted to the new 8.1 version", again as of 2005.

I know there's a cost to advancing the cutoff, in case there are
many PL/Java users still nursing along 8.0.3 databases, but there's
also a disincentive for interested contributors if we don't, because
it's really hard to develop new code with a hope of it also working
in ten-year-old EOLed environments, and it limits what can be aspired to.

As regards Java, we already have a component (the annotation-driven
SQL generator) that requires 1.6 or newer. 1.5 is pretty old now, and
before that, there were no generics. 1.7 brings try-with-resources,
which can make JDBC coding much less tedious.

In PostgreSQL:

8.1 added SET ROLE; PL/Java still only understands the three IDs
a session had before 8.1 (originally-authenticated ID, session ID,
and the effective one during a security-definer function), not
the four IDs there have been since 8.1. That leads to strangeness
like:

You are connected as user "bob".
=> SET ROLE schemaowner;
SET ROLE
=> CREATE TABLE foostuff.test;
CREATE TABLE
=> SELECT sqlj.install_jar('foo.jar', 'foo', true);
ERROR: permission denied for schema foostuff

... because PL/Java's install_jar (quite properly) dropped its own
privilege before running the deployment descriptor, but it dropped
back to "bob" instead of "schemaowner".

If we advance the backward-compatibility bar at least as far as 8.1,
we can fix that without having to special-case it.

8.2 added the RETURNING clause on DML statements. That is handy
because PL/Java's built-in JDBC implementation hasn't ever had
the special methods that retrieve generated IDs from a DML statement,
but as long as the backend groks RETURNING there's a perfectly usable
alternative (arguably even easier to use and to read).

8.4 added WITH, great for writing readable testcases!

WITH
ref AS (VALUES ('adjective', 'avaricious'), ('noun', 'platypus')),
test AS (SELECT * FROM javatest.propertyExampleAnno()),
missing AS (TABLE ref EXCEPT TABLE test)
SELECT 0 = count(*) FROM missing;

8.4 is also the latest to go EOL (about a year ago).

9.1 brought the extension-packaging system, which could offer a
target for PL/Java's various deployment methods to converge on.
One thing it can do is automatically select and run schema-updating
scripts, which is an outstanding PL/Java issue (#12).

9.1 goes EOL just over a year from now:
http://www.postgresql.org/support/versioning/

So, by biting the bullet and declaring some reasonable newer cutoffs
for back-compatibility, the project can become more contributor-
friendly.

STREAMLINING INSTALLATION/DEPLOYMENT

As PL/Java has been getting built, there is one part of installation
that doesn't quite fit with the others. *Almost* everything about
PL/Java gets configured using PostgreSQL's own mechanisms (in
postgresql.conf for dynamic_library_path and pljava.classpath, with
sqlj.set_classpath for user classes) but the location of libjvm itself
has not been. The native library has been built with a reference to
libjvm baked in, which the OS tries to find before PostgreSQL settings
can have any effect, and when it fails you have to pick one of the
solutions from the wiki that are all system-dependent and outside of
PostgreSQL (from the "installing on linux" page, edit LD_LIBRARY_PATH
in a PostgreSQL startup script, or tweak /etc/ld.conf.so, or, from
the "building pljava" page, figure out the right link options to build
the native library with an RPATH). The RPATH one is my favorite, but
only because it's the only one that isn't tweaking some shared system
state just to make one thing work. But it's still a pain, especially
rebuilding with a new RPATH if the JRE location changes.

So I've started a branch where you also set the libjvm location in
postgresql.conf like anything else, using pljava.libjvm_location.

https://github.com/jcflack/pljava/tree/feature/master/installhelp

I haven't made it a pull request yet because it's the sort of change
that might well be tested on a few other platforms besides mine.
I don't expect many portability problems with the dynamic loading,
because I used PostgreSQL's own pg_dlopen wrapper, which the main
project has already done the heavy lifting of getting to work on
at least nine different platforms. Any platform where pg_dlopen won't
work is one where PL/Java won't work anyway. I simply changed the maven
project to not bake the libjvm reference into the pljava object,
and pljava's native code can now get control and load libjvm using
the configured pljava.libjvm_location. If it fails, it can even give
a useful error with a hint to SET pljava.libjvm_location.

That's where it stands now but it doesn't have to stop there. With an
assign hook on the variable, it could even pick up where it left off
after a SET pljava.libjvm_location command. It could do the same with
pljava.classpath if the jar isn't found, and then even complete the SQL
installation steps if the objects aren't there yet. It could then
log those working settings and remind you to add them into
postgresql.conf, or (in 9.4+) even use ALTER SYSTEM to do that directly.

As a result, it could be possible to quickly try out PL/Java without
editing any configs at all: just start with

LOAD /path/to/pljava.so;

and then issue SET pljava.libjvm_location and SET pljava.classpath
commands as the messages suggest, and watch it go.

Anyway, those are my current ideas on the state of PL/Java and the
parts of it I am working on. I'm sorry it came out so long, but
I wanted to get several of the ideas out to the list for comment
(at least from those of you who'll read something this long. :)
For those who won't maybe I'll do some later messages taking up
smaller pieces individually.

Cheers,
-Chap