Random disconnects

Hi All,

Just wondering whether someone might recognize the problem i am
having. First the intro: postgresql-8.1.0 running on FreeBSD
6.0-RELENG, on a dual xeon 3.0GHZ machine with 2gb memory/2gb swap.

The problem is we have been experiencing random disconnects from our
main application and a script. The application is a web spider
written in java that uses the postgresql db to store urls, and quite
a lot of them. The script is a perl script that coordinates several
of these crawlers, and has a low postgresql utilisation.

On the machines with the java app (there are two of them currently,
each with its own postgresql database) we use the latest jdbc:

postgresql-8.1-405.jdbc3.jar

So the errors we see are:

In java, this big fat exception:

org.postgresql.util.PSQLException: An I/O error occured while
sending to the backend.
Call stack:
org.postgresql.util.PSQLException: An I/O error occured while
sending to the backend.
at
org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:214)
at
org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:437)
at
org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:353)
at
org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate(AbstractJdbc2Statement.java:307)
at
net.oneworld.issuemapper.CrawlToDBListener.dumpLinkToSpoolAndFreeUpMemory(CrawlToDBListener.java:214)
at
net.oneworld.issuemapper.CrawlToDBListener.crawled(CrawlToDBListener.java:165)
at websphinx.Crawler.sendLinkEvent(Crawler.java:1540)
at websphinx.Crawler.process(Crawler.java:1910)
at websphinx.Crawler.fetch(Crawler.java:1840)
at websphinx.Worm.run(Crawler.java:1994)
Caused by: java.net.SocketException: Network is unreachable
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
at
java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:66)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:124)
at org.postgresql.core.PGStream.flush(PGStream.java:502)
at
org.postgresql.core.v3.QueryExecutorImpl.sendSync(QueryExecutorImpl.java:672)
at
org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:187)
... 9 more

Where at CrawlToDBListener.java:214 we just execute a prepared
statement that inserts the url data, which goes allright 99.99999%
of the time.

The perl error we get is:

DBI connect('dbname=issuemaps_a;host=192.168.0.2','crawler',...)
failed: server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
at cbsd-lib.pl line 181
Can't call method "prepare" on an undefined value at cbsd-lib.pl
line 188, <SIZEFILEH> line 68.

Line 188 of cbsd-lib is a dbh->prepare line..

I guess the proper work-around is to wrap all database calls such
that when it detects a loss of connection, it will retry (which is
what i would've done in the first place if i was the one who had
written this software), or use dbh / jdbc persistent connections to
make this transparent.

But I think there might be some underlying problem and if so, i'd
like to know about it and fix it.

Gr,

Koen

--
K.F.J. Martens, Sonologic, http://www.sonologic.nl/
Networking, hosting, embedded systems, unix, artificial intelligence.
Public PGP key: http://www.metro.cx/pubkey-gmc.asc
Wondering about the funny attachment your mail program
can't read? Visit http://www.openpgp.org/