Renovating the PL/Java build process

From: Kartik Ohri <kartikohri13(at)gmail(dot)com>
To: pljava-dev(at)lists(dot)postgresql(dot)org
Subject: Renovating the PL/Java build process
Date: 2020-08-31 08:12:30
Message-ID: CAASLQ4OKtpbTcxDJTg6vABVuhPhGAwYbgrsNbsYQADPkkPiwJw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pljava-dev

It is the time of the summer when GSoC participants look back on the work
done in the past few months and so here are we. The original project idea
and the project proposal had three major components. Each of these has been
discussed in detail below.

1) Continuous Integration

Expected project outcome: CI builds and reports occur on pull requests, for
multiple versions of PostgreSQL and Java, and on Linux, macOS, and Windows.

Linked Pull Request: https://github.com/tada/pljava/pull/289

State of the work: The work has been largely successful. PL/Java now has a
CI pipeline built using Travis CI and AppVeyor CI. In a repository where
these CI providers have the required access, every commit and PR are built
and tested on over 30 combinations of PostgreSQL and Java versions across
Linux, Mac OS X and Windows (MSVC and MinGW). This has already begun to
bear fruit as can be seen from https://github.com/tada/pljava/issues/282
and https://github.com/tada/pljava/issues/274.

Scope for future work: However, there is scope of several improvements. The
CI infrastructure can be moved to Github Actions. The motive is to
eliminate unnecessary access to external providers. Also, the test suite
can be made more comprehensive by adding builds from PostgreSQL source on
Mac OS X and Windows platform as well.

2) PL/Java API Artifact

Expected project outcome: pljava-api jar artifact deployed to Maven Central
when a new release is made.

State of the Work: The component has not been satisfied yet. However, the
mentor is satisfied with the progress since there is no new artifact ready
for release yet. It is our consensus that the introduction of PL/Java PGXS
Plugin and CI service providers has gone a long way in this respect. The
task should be relatively easy to accomplish now.

Scope for future work: Add a deploy phase to either of the CI providers to
publish the pljava-api artifact.

3) Replacing nar-maven-plugin

Expected project outcome: Build system no longer uses nar-maven-plugin,
correctly uses build settings from pg_config, and is tested on at least
Linux, macOS, and Windows (with Visual Studio and with MinGW).

Linked Pull Request:

https://github.com/tada/pljava/pull/288

https://github.com/tada/pljava/pull/292 (commit SHA #1df123b
<https://github.com/tada/pljava/pull/292/commits/1df123b8208272b54ca6c62978fe0c7acc17a3da>
)

State of the Work: The PL/Java PGXS plugin has successfully retired
nar-maven-plugin.
The CI builds for the project being built using the PGXS plugin are
passing. The PGXS plugin also queries pg_config to obtain various flags and
passes them on to the compiler and the linker. Overall, the plugin is in a
functional state and works just fine. Several useful utilities are also
exposed by the plugin for using in the scripts.

PL/Java PGXS is a complete rework of the earlier build process in a true
sense. It addresses many pain points that the nar maven plugin had
introduced. The biggest issue posed by nar maven was highly noisy builds.
Finding the actual errors among the tons of false positives was no easier
than finding a needle in a haystack. To make matters worse, the debugging
information provided by the nar plugin was average at best. For instance,
many times the build would fail with an error in include paths. Ok fine, let's
see why the path was incorrect. But no, you can only see an id assigned to
the include path by the JVM. Good luck debugging with that.

Another pressing issue is that nar tries to abstract a lot of details,
hence compromising code readability. The actual arguments and configuration
details passed to the compiler and linker are hidden deep inside nar maven's
source code. On the other hand, the pljava-so pom.xml file had a huge block
of javascript to just quote a string correctly for passing to the C code. The
point is not to berate nar maven plugin. It is a great plugin, but it is
just not suited for our simple use case.

Enter PL/Java PGXS. PL/Java PGXS turns this completely inside out. It hides
the uninteresting bland details and puts the important configuration
details in the pom. PL/Java PGXS is specifically geared towards building
PL/Java, hence unnecessary abstractions can be done away with.

The PGXS plugin has also retired the troublesome maven-javadoc-plugin and
supports building reports and scripting during the site phase.
maven-javadoc-plugin
tries to micromanage a lot of stuff when just running the javadoc on a set
of files is enough to do the right job. The current implementation using
PGXS plugin just does that by utilizing the APIs exposed by Java itself.

Scope for Future Work: Since PGXS plugin supports scripting, there is no
need for using maven antrun plugin for the same. The necessary code can be
ported from using maven antrun plugin to use PGXS instead. Further, the
exported symbols can be filtered to expose only PL/Java entry points. In
addition, currently the output produced by the compiler and linker is
redirected to standard output and standard error streams. It would be
better if the output was rather piped through the plugin in which case it
could be filtered and utilised as desired.

Responses

Browse pljava-dev by date

  From Date Subject
Next Message Thomas Hallgren 2020-08-31 11:33:33 Re: Travis and AppVeyor continuous integration [Re: feature/master/ci]
Previous Message Chapman Flack 2020-08-30 12:46:36 Re: Travis and AppVeyor continuous integration [Re: feature/master/ci]