Avatica

We are proposing to build an open source database connectivity framework called Avatica. This may seem a strange request to post to a Postgres list, but please bear with us.

The most exciting database and data management systems these days are open source. Many of those, especially those coming from the Hadoop ecosystem, are written in Java. Many of the clients to those systems are in Java, but there are still plenty of important clients that require an ODBC client. The list includes applications written in C, C++, .NET, perl, php or python, and BI tools such as Tableau and MicroStrategy.

But paradoxically, the major open source databases written in Java do not have open source ODBC drivers. How can that be? When an open source database reaches a certain level of market success, its parent company releases a commercial ODBC driver. The driver is often free, as in beer, the market demand is satisfied, and an open source driver never happens.

As you know, having a fully open source stack is a good thing. Not least, it enables innovation. Other projects can improve pieces of the stack but leave the rest intact. The maintenance work can be spread among many developers, which results in higher quality software for all.

With this in mind, we are proposing Avatica. Avatica consists of an ODBC driver, an RPC protocol, an RPC server written in Java, and a Java SPI to be implemented by a particular database [1], [2].

For a Java-based database, it just needs to implement the Java SPI describing its identity (name and version), behavior (for example, whether transactions are supported, and SQL compliance level), metadata (list of tables and columns) and basic session operations (create and close connections, prepare, execute, fetch and close statements). Given that information, the ODBC driver just works. As much as possible is done on the server side, so changes so the ODBC driver are rare.

This is not a small task. We know how hard it is to build a quality ODBC driver, and to establish a community to maintain it long-term.

The initial projects using Avatica would be Apache Drill, Apache Phoenix, and Optiq. They will bring growing bases of users and developers, and resources to build the various pieces. If Avatica is successful, other projects will undoubtedly join them.

We would like to start this project by building out from psqlODBC. From our perspective, it makes sense to start with a proven, portable open source ODBC driver, and psqlODBC is the best out there. But we believe that Avatica will bring psqlODBC to a new audience, new Big Data challenges, and bring on board new communities of users and developers.

Is this an interesting project for the psqlODBC community? If so, let us know how we can work together.

Julian Hyde (Optiq, Mondrian)
James Taylor (Apache Phoenix)
Jacques Nadeau (Apache Drill)

[1] Avatica architecture https://raw.github.com/julianhyde/share/master/slides/avatica-architecture.png
[2] Avatica proposal https://raw.github.com/julianhyde/share/master/slides/avatica-proposal.pdf