Re: Trouble with COPY IN

On Tue, 20 Jul 2010, Matthew Wakeling wrote:
> So, should the unlocking happen elsewhere, or should endCopy not require a
> lock?

Or a third option. We should delay the processing of the "Z" (Ready for
query) message until after the endCopy has been called.

I have still not been able to create a test that provokes this bug. I do
not understand under what conditions the server will do this, but
sometimes it will respond with a "C" and a "Z" before the endCopy message
has actually been sent to it. Maybe it is keying off the "-1" (no further
rows) part of the binary copy. In any case, I am including a patch here
which seems to fix the driver for my application. When the driver receives
a "C" message (command finished), and block is set to false, then we can
assume that a little later on we will call processCopyResults again very
soon with block set to true. Instead of setting block to true, finish
processing of messages in order to delay seeing the "Z" message until
after the endCopy method has been called.

The patch also contains a simple test for binary copy in, although it does
not exercise this bug.

Index: org/postgresql/core/v3/QueryExecutorImpl.java
===================================================================
RCS file:
/cvsroot/jdbc/pgjdbc/org/postgresql/core/v3/QueryExecutorImpl.java,v
retrieving revision 1.48
diff -u -r1.48 QueryExecutorImpl.java
--- org/postgresql/core/v3/QueryExecutorImpl.java 21 Mar 2010 07:13:37 -0000 1.48
+++ org/postgresql/core/v3/QueryExecutorImpl.java 20 Jul 2010 14:48:03 -0000
@@ -931,7 +931,9 @@
error = se;
}

- block = true;
+ if (!block) {
+ endReceiving = true;
+ }
break;

case 'E': // ErrorMessage (expected response to CopyFail)
Index: org/postgresql/test/jdbc2/CopyTest.java
===================================================================
RCS file: /cvsroot/jdbc/pgjdbc/org/postgresql/test/jdbc2/CopyTest.java,v
retrieving revision 1.4
diff -u -r1.4 CopyTest.java
--- org/postgresql/test/jdbc2/CopyTest.java 8 Apr 2010 16:18:05 -0000 1.4
+++ org/postgresql/test/jdbc2/CopyTest.java 20 Jul 2010 14:48:03 -0000
@@ -15,6 +15,7 @@
import java.sql.Statement;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
+import java.io.DataOutputStream;
import java.io.InputStream;
import java.io.OutputStream;
import java.io.PrintStream;
@@ -272,4 +273,35 @@
assertEquals(0, getCount());
}

+ public void testBinaryCopyIn() throws Exception {
+ con.setAutoCommit(true);
+ String sql = "COPY copytest FROM STDIN BINARY";
+ ByteArrayOutputStream data = new ByteArrayOutputStream();
+ DataOutputStream dos = new DataOutputStream(data);
+ dos.writeBytes("PGCOPY\n");
+ dos.writeByte(255);
+ dos.writeBytes("\r\n");
+ dos.writeByte(0); // Signature done
+ dos.writeInt(0); // Flags - we aren't supplying OIDS
+ dos.writeInt(0); // Length of header extension
+ for (int row = 0; row < 99700; row++) {
+ dos.writeShort(3); // Three rows
+ dos.writeInt(5); // String length
+ dos.writeByte('H');
+ dos.writeByte('e');
+ dos.writeByte('l');
+ dos.writeByte('l');
+ dos.writeByte('o');
+ dos.writeInt(4); // int length
+ dos.writeInt(row); // Second argument
+ dos.writeInt(-1); // Null third argument
+ }
+ dos.writeShort(-1); // End of rows
+ dos.flush();
+ dos.close();
+ byte[] byteArray = data.toByteArray();
+ //System.err.println("Data length: " + byteArray.length);
+ long count = copyAPI.copyIn(sql, new ByteArrayInputStream(byteArray));
+ assertEquals(99700, count);
+ }
}

Hope this helps,

Matthew

--
There once was a limerick .sig
that really was not very big
It was going quite fine
Till it reached the fourth line