Connection attempt hangs when server is unresponsive

We are using 1.2.1 of DBCP with our tomcat server to pool connections to
databases on the wide area. We had an issue today were our tomcat threads
started increasing dramatically when we had a Postgres server go away and we
found a lot of threads stuck trying to create a datasource:

at
org.apache.commons.dbcp.BasicDataSource.createDataSource(BasicDataSource.jav
a:753)
at
org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:5
40)
at
net.iglass.db.JdbcDbAccess.createConnection(JdbcDbAccess.java:122)
at
net.iglass.db.JdbcDbAccess.createConnection(JdbcDbAccess.java:134)

Since createDataSource is synchronized we traced them to the following
thread that had them blocked:

at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
at
java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:516)
at java.net.Socket.connect(Socket.java:466)
at java.net.Socket.(Socket.java:366)
at java.net.Socket.(Socket.java:179)
at org.postgresql.core.PGStream.(PGStream.java:58)
at
org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFa
ctoryImpl.java:77)
at
org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:
65)
at
org.postgresql.jdbc2.AbstractJdbc2Connection.(AbstractJdbc2Connection.java:1
17)
at
org.postgresql.jdbc3.AbstractJdbc3Connection.(AbstractJdbc3Connection.java:3
0)
at org.postgresql.jdbc3.Jdbc3Connection.(Jdbc3Connection.java:24)
at org.postgresql.Driver.connect(Driver.java:235)
at
org.apache.commons.dbcp.DriverConnectionFactory.createConnection(DriverConne
ctionFactory.java:37)
at
org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnect
ionFactory.java:290)
at
org.apache.commons.dbcp.BasicDataSource.validateConnectionFactory(BasicDataS
ource.java:877)
at
org.apache.commons.dbcp.BasicDataSource.createDataSource(BasicDataSource.jav
a:851)
at
org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:5
40)
at
net.iglass.db.JdbcDbAccess.createConnection(JdbcDbAccess.java:122)
at
net.iglass.db.JdbcDbAccess.createConnection(JdbcDbAccess.java:134)

My question is why this thread did not error out when it could not talk to
the server? Could the server have been up when the connection started and
went away in the middle causing errors? Could there be something we have
overlooked in our conf that would time it out? When the db server came
back, our thread count dropped from 200 down to 35 which is normal.

Thanks in advance,
Woody

----------------------------------------
iGLASS Networks
211-A S. Salem St
Apex NC 27502
(919) 387-3550 x813
www.iglass.net