From: | "David G(dot) Johnston" <david(dot)g(dot)johnston(at)gmail(dot)com> |
---|---|
To: | Alex Ignatov <a(dot)ignatov(at)postgrespro(dot)ru> |
Cc: | Scott Marlowe <scott(dot)marlowe(at)gmail(dot)com>, "pgsql-general(at)postgresql(dot)org" <pgsql-general(at)postgresql(dot)org> |
Subject: | Re: Silent data loss in its pure form |
Date: | 2016-05-30 20:42:09 |
Message-ID: | CAKFQuwbnZVJORTvTg_Mhn=+pxX3-K6M1HyxAPhTD+WFnBgi33Q@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-general |
On Mon, May 30, 2016 at 4:22 PM, Alex Ignatov <a(dot)ignatov(at)postgrespro(dot)ru>
wrote:
>
> _____________________________
> From: Scott Marlowe <scott(dot)marlowe(at)gmail(dot)com>
> Sent: Monday, May 30, 2016 20:14
> Subject: Re: [GENERAL] Silent data loss in its pure form
> To: Alex Ignatov <a(dot)ignatov(at)postgrespro(dot)ru>
> Cc: <pgsql-general(at)postgresql(dot)org>
>
>
>
> On Mon, May 30, 2016 at 10:57 AM, Alex Ignatov <a(dot)ignatov(at)postgrespro(dot)ru>
> wrote:
> > Following this bug reports from redhat
> > https://bugzilla.redhat.com/show_bug.cgi?id=845233
> >
> > it rising some dangerous issue:
> >
> > If on any reasons you data file is zeroed after some power loss(it is the
> > most known issue on XFS in the past) when you do
> > select count(*) from you_table you got zero if you table was in one
> > 1GB(default) file or some other numbers !=count (*) from you_table before
> > power loss
> > No errors, nothing suspicious in logs. No any checksum errors. Nothing.
> >
> > Silent data loss is its pure form.
> >
> > And thanks to all gods that you notice it before backup recycling which
> > contains good data.
> > Keep in mind it while checking you "backups" in any forms (pg_dump or the
> > more dangerous and short-spoken PITR file backup)
> >
> > You data is always in danger with "zeroed data file is normal file"
> > paradigm.
>
> That bug shows as having been fixed in 2012. Are there any modern,
> supported distros that would still have it? It sounds really bad btw.
>
>
> --
> Sent via pgsql-general mailing list (pgsql-general(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-general
>
> It is not about modern distros it is about possible silent data loss in
> old distros. We have replication, have some form of data check summing, but
> we are powerless in front of this XFS bug just because "zeroed file is you
> good friend in Postgres".
> With "zero file is good file" paradigm and this noted XFS bug PG as it
> is now is "colossus with feet of clay" It can do many things but it cant
> even tell us that we have some trouble with our precious data.
> No need to prevent or to some other AI magic and so on when zero doom day
> has come.
> What we need now is some error report about suspicious zeroed file. To
> make us sure that something went wrong and we have to do recovery.
> Today PG "power loss" recovery and this XFS bug poisoning our ensurance
> that recovery went well . It went well even with zeroed file. It it not
> healthy behavior. It like a walk on a mine field with eyes closed.
> I think it is very dangerous view on data to have data files without any
> header in it and without any files checking at least on PG start.
> With this known XFS bug it can leads to undetected and unavoidable loss
> of data.
>
For those not following -general this is basically an extension of the
following thread.
"Deleting a table file does not raise an error when the table is touched
afterwards, why?"
David J.
From | Date | Subject | |
---|---|---|---|
Next Message | David G. Johnston | 2016-05-30 20:45:27 | Re: Deleting a table file does not raise an error when the table is touched afterwards, why? |
Previous Message | Andreas Joseph Krogh | 2016-05-30 20:33:54 | Re: Slides for PGCon2016; "FTS is dead ? Long live FTS !" |