Bug #948: Postgresql server abruptly shutdown now database doesnt start

Mandar Salunkhe (mandar(dot)salunkhe(at)hp(dot)com) reports a bug with a severity of 1
The lower the number the more severe it is.

Short Description
Postgresql server abruptly shutdown now database doesnt start

Long Description
We have been using Postgresql for the past 6 months.
Yesterday due to power shutdown the server on which Postgresql database resides was abruptly shutdown.
Now when we try to start it does not start and just hangs. We are running on a Win2k with Postgresql 7.2 with cygwin install, version 2.125.2.10

This is the startup process we run.
On cygwin

Administrator(at)HPA4W357 ~
$ cd /usr/local/pgsql/bin/

Administrator(at)HPA4W357 /usr/local/pgsql/bin
$ ipc-daemon &
[1] 840

Administrator(at)HPA4W357 /usr/local/pgsql/bin
$ ./postmaster -i -D /usr/local/pgsql/data

Normally when we start in this manner, we get

DEBUG: database system was shut down at 2003-04-16 12:14:24
DEBUG: checkpoint record is at 0/5F8778
DEBUG: redo record is at 0/5F8778; undo record is at 0/0; shutdown TRUE
DEBUG: next transaction id: 9774; next oid: 18504
DEBUG: database system is ready
( we have given this output from a machine where we have another install of Postgresql)

But now when we run this , we get

hp(at)ACACIA ~
$ cd /usr/local/pgsql/bin/

hp(at)ACACIA /usr/local/pgsql/bin
$ ipc-daemon &
[1] 18140

hp(at)ACACIA /usr/local/pgsql/bin
$ ./postmaster -i -D /usr/local/pgsql/data

And the process hangs there.

When we check the status of Postgresql using

hp(at)ACACIA ~
$ /usr/local/pgsql/bin/pg_ctl -D /usr/local/pgsql/data status
pg_ctl: postmaster is running (pid: 18012)
Command line was:
/usr/local/pgsql/bin/./postmaster '-i' '-D' '/usr/local/pgsql/data'

The data directory shows:
hp(at)ACACIA /usr/local/pgsql/data
$ ls -lrt
total 28
-rw-r--r-- 1 hp None 5 Mar 13 03:09 PG_VERSION
drwxr-xr-x 2 hp None 0 Mar 13 03:09 pg_clog
-rw-r--r-- 1 hp None 4027 Mar 13 03:09 postgresql.conf
-rw-r--r-- 1 hp None 1285 Mar 13 03:09 pg_ident.conf
-rw-r--r-- 1 hp None 10313 Mar 13 03:18 pg_hba.conf
drwxr-xr-x 5 hp None 0 Apr 5 17:00 base
-rw-r--r-- 1 hp None 69 Apr 7 23:17 postmaster.opts
drwxr-xr-x 2 hp None 4096 Apr 9 01:06 pg_xlog
drwxr-xr-x 2 hp None 4096 Apr 15 09:49 global
-rw-r--r-- 1 hp None 30 Apr 17 20:50 postmaster.pid

So the postmaster.pid file is present. When we remove the pid file, even then,

hp(at)ACACIA ~
$ /usr/local/pgsql/bin/pg_ctl -D /usr/local/pgsql/data status
pg_ctl: postmaster or postgres is not running

hp(at)ACACIA ~
$ cd /usr/local/pgsql/bin/

hp(at)ACACIA /usr/local/pgsql/bin
$ ipc-daemon &
[1] 18080

hp(at)ACACIA /usr/local/pgsql/bin
$ ./postmaster -i -D /usr/local/pgsql/data

The process still hangs.
Also when we check the status using
hp(at)ACACIA ~
$ /usr/local/pgsql/bin/psql SGDB

This command hangs too.

Viewing the pid file ( newly created ) shows:
18140
/usr/local/pgsql/data

And then

hp(at)ACACIA /usr/local/pgsql/data
$ ps -aef|grep 18140
hp 18140 18204 con 21:19:40 /usr/bin/ps

There is a file in /tmp called .s.PGSQL-5432 which we removed and tried the same process. But no use. ( At this stage we were really desperate and trying out things which we picked from the net)

Is there a solution to this problem? We would really appreciate to get the solution.
Please help as this is our production server and we are stuck.

Thanks

Mandar Salunkhe

Sample Code

No file was uploaded with this report