Re: Batches of single-insert statements vs batches of multi-insert statements

From: Christopher Deckers <chrriis(at)gmail(dot)com>
To: Jeremy Whiting <jwhiting(at)redhat(dot)com>
Cc: pgsql-jdbc(at)postgresql(dot)org
Subject: Re: Batches of single-insert statements vs batches of multi-insert statements
Date: 2016-06-08 13:15:40
Message-ID: CADFnS4STejMLgYsmtAdhRrU-EJ3z-Q7Uun1wjrdSah+KWupMkw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-jdbc

Hi Jeremy,

> I suggest you avoid trying to hard coding data values in the statement
> sql.
>

The resulting SQL is not hand-crafted, and is built through all sorts of
layers depending on various rules. The end result that I exposed does not
tell how it was built.

Note that from a theoretical point of view, if I were to add parameters for
fixed values, then I would send through the wire as many times this
parameter, while if it is part of the statement the driver can decide to
only send the dynamic parameters. Also, we do have external multi-insert
logic for that specific part of the code, and it also improves speed with
jTDS; but SQL Server is limited to 2000 dynamic parameters, so if we waste
those parameters with static values, less batches can go through at once.

In any case, it must be supported by the batch rewrite feature, whether
people like the style or not :)

> c) Provide a patch to add implement sql statement parsing support for hard
> coded data. I see you have already started to try this [2].
>

In fact, the latest version of my patch hooks into the existing query
parsing, and just has to record the start and end of the values block. That
combined with the positions of the parameters made the latest version
reliable.
That being said, there is currently no safety in case we put too many
batches overflowing Postgres limits, which is what I will try to work on
when I get some free time.

Cheers,
-Christopher

In response to

Browse pgsql-jdbc by date

  From Date Subject
Next Message Campbell, Lance 2016-06-08 15:35:28 How to properly convert PostgreSQL timestamp to Java xsd:dateTime
Previous Message Jeremy Whiting 2016-06-08 12:46:40 Re: Batches of single-insert statements vs batches of multi-insert statements