Re: Losing data

-------------- Original message ----------------------
From: Garry Saddington <garry(at)schoolteachers(dot)co(dot)uk>
> On Thursday 19 June 2008 18:52, Adrian Klaver wrote:
> > -------------- Original message ----------------------
> > From: Garry Saddington <garry(at)schoolteachers(dot)co(dot)uk>
> >
> > > On Thursday 19 June 2008 16:55, Joshua D. Drake wrote:
> > > > On Thu, 2008-06-19 at 16:55 +0100, Garry Saddington wrote:
> > > > > I have had a serious loss of data and wondered if anyone could shed
> > > > > any light on what may have happened.
> > > > > My users have been writing reports on students. No error messages
> > > > > have been produced and when called back up the reports seem to be
> > > > > present at the time of writing. However, next day they have
> > > > > disappeared, and they do not appear in a pg_dump. They seem to have
> > > > > been kept in memory and never written to disk.
> > > > > We are using Zope and connecting to Postgres through psycopg on
> > > > > Centos 5. I suspect a hard disk failure but any other ideas would be
> > > > > welcome. Would these reports be in the WAL?
> > > >
> > > > If it was hardware related you would know, quickly. This sounds a great
> > > > deal more like an application level interaction. Perhaps your zope
> > > > application caches things for a while before committing to disk?
> > >
> > > Yes I thought of this but once the report is sent to the DB a separate
> > > query is run to get all of that teacher's reports and these are then
> > > displayed on a new page. They all appear here but then disappear later.
> > > Zope has transaction machinery that rolls everything back on an error, so
> > > Postgres must have indicated a successful write somehow. I read in a
> > > Postgres manual that the hard disk may report to the OS that a write has
> > > occured when it actually has not, is this possible? Oh, and the problem
> > > has been intermittant. Another thing that happened this morning is that
> > > Postgres had today as 18/06/2008 when in fact it was 19/06/2008 and the
> > > OS reported this correctly. Restarting postgres sorted it, could this be
> > > the problem?
> > > Regards
> > > Garry
> >
> > Seems like a transaction with no commit. Basically along as the session is
> > active the data is there but once the session is closed the data does not
> > persist.
> >
> Makes sense but what is to blame?

Therein lies the rub. As Josh mentioned in another post you will need turn up the logging for the various components and rerun the suspect procedure. I find the Postgres logs to most informative as they show what is actually hitting the database. Once I see the actual SQL statement I work backwards to see how it got there and in that form.

> Regards
> Garry

--
Adrian Klaver
aklaver(at)comcast(dot)net