Re: [BUGS] BUG #1347: Bulk Import stopps after a while (

Kris Jurka wrote:
> On Wed, 15 Dec 2004, Oliver Jowett wrote:
>
>>OK, it looks like it is the connection deadlock issue, but triggered by
>>a case I hadn't considered. For some reason every query in the batch is
>>creating a new named statement on the backend .. and the statement
>>cleanup process that happens before query execution is flooding
>>statement Close commands down the connection on the second batch
>>execution, which appears to deadlock the connection. The exact behaviour
>>will be fairly unpredicatable as it depends on when the JVM does garbage
>>collection.
>
> Any news on this?

I didn't get a chance to look at it before going on leave. I
won't be able to look at it before early January.

> In general it seems like the whole statement/portal
> cleanup is too lazy. These things can take up significant backend
> resources and I would have expected calling ResultSet.close() and/or
> Statement.close() to actually close something (perhaps not immediately,
> but at least on the next network trip).

I don't have the code to hand, but from memory an explicit
close of the Query does exactly this -- it clears the
reference which causes it to be immeditately enqueued, and
the next query will do cleanup when it polls the reference
queue.

The GC-driven enqueue only kicks in if the object holding
the statement info becomes garbage without being closed.

The reference queue stuff seems to be working as designed.
The problem is that we should be taking the Close messages
we send into account when trying to avoid protocol
deadlocks, but currently we don't, so a big batch of Close
messages at once break things.

I was more worried about why a batch update was generating
so many individual named statements.. it should either use
zero (Statement.addBatch) or one
(PreparedStatement.addBatch) named statement, not N statements.

-O