Re: [HACKERS] Pipelining executions to postgresql server

> On Nov 4, 2014, at 12:55 AM, Craig Ringer <craig(at)2ndquadrant(dot)com> wrote:
>
> On 11/04/2014 07:56 AM, Mikko Tiihonen wrote:
>> I also think the async I/O is the way to go. Luckily that has already been done
>> in the pgjdbc-ng (https://github.com/impossibl/pgjdbc-ng) built on top
>> of netty java NIO library. It has quite good feature parity with the original
>> pgjdbc driver.
>
> Huh, it does seem to now. The big omission, unless I'm blind, is support
> for COPY. (It also lacks support for JDBC3 and JDBC4, requiring JDK 7
> and JDBC 4.1, but that's reasonable enough.)
>

I focused on 4.1 compliance instead of legacy support. I am proud to say I believe pgjdbc-ng is 100% feature compliant for JDBC 4.1. Legacy support is still not, and likely will never be, planned.

It does lack COPY support as I have never used it myself from the driver, only from tools outside the JVM.

> It now has LISTEN/NOTIFY by the looks, and of course it's built around
> binary protocol support.
>
> I freely admit I haven't looked at it too closely. I'm a tad frustrated
> by the parallel development of a different driver when the "official"
> driver has so much in the way of low hanging fruit to improve.
>

I’ll only nibble this bait…. I outlined my reasons when I started the project. They were all valid then and still are now. My apologies for causing any frustration with this new path. It’s a lot cleaner, simpler and provides more JDBC features than the original driver because of it. Although I must say, without the original driver and it’s exhausting battery of unit tests, building a new driver would seem impossible.

> I have to say I like some aspects of how pgjdbc-ng is being done - in
> particular use of Maven and being built around non-blocking I/O.
>
> OTOH, the use of Google Guava I find pretty inexplicable in a JDBC
> driver, and since it hard-depends on JDK 7 I don't understand why Netty
> was used instead of the async / non-blocking features of the core JVM.
> That may simply be my inexperience with writing non-blocking socket I/O
> code on Java though.
>

It confuses me as to why you consider using stable, well implemented, well tested and well cared for libraries as inexplicable. Just because we are building a “driver” means we have to write every line of code ourselves? No thanks. You can imagine our differences on this philosophy are one of the reasons why I consider pgjdbc-ng’s parallel development to be a godsend rather than hacking on the original code.

Concerning Guava… A great library with an amazing number of classes that make everyday Java easier. The requirement for JDK 7 was chosen before Guava was considered not because of it. Using it seemed obvious after that decision. Also, we have internalized the classes we use out of Guava to remove it as a dependency. This is more work to maintain on our part but makes it worth it when deploying a single JAR.

Concerning Netty… All options were entertained at the beginning. The original version actually used a basic NIO socket. After I realized I had to basically write my own framework to work with this socket correctly I searched for an alternative and found Netty. The deciding factor was that Implementing SSL on on top of the NIO API was considered next to impossible to get right; according to all prevailing wisdom at the time. Whereas with Netty, SSL support is basically a single line change.

> I'm also concerned about how it'll handle new JDBC versions, as it seems
> to lack the JDBC interface abstraction of the core driver.
>

My plan is to handle adding support for 4.2 and beyond by using a Maven based conditional preprocessor. If that fails, or seems just too ugly, I’ll probably have to look at an abstract class based method like that of the original driver.

>> I do not think the JDBC batch interface even allow doing updates to multiple
>> tables when using prepared statements?
>
> Ah, I missed this before.
>
> That's correct. You get prepared statements _or_ multiple different
> statements.
>
> That's a more understandable reason to concoct a new API, and explains
> why you're not just solving the issues with batches. Though I still
> think you're going to have to fix the buffer management code before you
> do anything like this.
>
> --
> Craig Ringer http://www.2ndQuadrant.com/
> PostgreSQL Development, 24x7 Support, Training & Services
>
>
> --
> Sent via pgsql-jdbc mailing list (pgsql-jdbc(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-jdbc