Re: Initdb --data-checksums by default

On 20.04.2016 12:10, Andreas Joseph Krogh wrote:
> På onsdag 20. april 2016 kl. 11:02:31, skrev Alex Ignatov
> <a(dot)ignatov(at)postgrespro(dot)ru <mailto:a(dot)ignatov(at)postgrespro(dot)ru>>:
>
> On 20.04.2016 11:40, Andreas Joseph Krogh wrote:
>> På onsdag 20. april 2016 kl. 10:33:14, skrev Alex Ignatov
>> <a(dot)ignatov(at)postgrespro(dot)ru <mailto:a(dot)ignatov(at)postgrespro(dot)ru>>:
>>
>>
>>
>> On 20.04.2016 11:29, Devrim Gündüz wrote:
>> > Hi,
>> >
>> > On Wed, 2016-04-20 at 10:43 +0300, Alex Ignatov wrote:
>> >> Today in Big Data epoch silent data corruption becoming
>> more and more
>> >> issue to afraid of. With uncorrectable read error rate ~
>> 10^-15 on
>> >> multiterabyte disk bit rot is the real issue.
>> >> I think that today checksumming data must be mandatory
>> set by default.
>> >> Only if someone doesn't care about his data he can
>> manually turn this
>> >> option off.
>> >>
>> >> What do you think about defaulting --data-checksums in initdb?
>> > I think this should be discussed in -hackers, right?
>> >
>> > Regards,
>> May be you right but i want to know what people think about
>> it before
>> i'll write to hackers.
>>
>> -1 on changing the default.
>> 10^15 ~= 1000 TB, which isn't very common yet. Those having it
>> probably are aware of the risk and have enabled checksums already.
>> --
>> *Andreas Joseph Krogh*
>> CTO / Partner - Visena AS
>> Mobile: +47 909 56 963
>> andreas(at)visena(dot)com <mailto:andreas(at)visena(dot)com>
>> www.visena.com <https://www.visena.com>
>> <https://www.visena.com>
> It is per bit not bytes. So it is ~100 TB. We working with some
> enterprise who have WALs creation rate ~ 4GB per min - so it is
> only max 100 days before you get bit rotted and have probability
> to get silent data corruption.
> Also don't forget that it is theoretical limit and Google tells us
> that HDD and SSD is not as reliable as manufactures tell. So this
> 10^-15 can easily be much higher.
>
> Ok, but still - the case you're describing isn't the common-case for
> PG-users. Enterprises like that certainly chould use --data-checksums,
> I'm not arguing against that, just that it shouldn't be the
> default-setting.
> --
> *Andreas Joseph Krogh*
> CTO / Partner - Visena AS
> Mobile: +47 909 56 963
> andreas(at)visena(dot)com <mailto:andreas(at)visena(dot)com>
> www.visena.com <https://www.visena.com>
> <https://www.visena.com>
Why do you think that common pg-users doesn't care about their data?
Also why do we have wal_level=minimal fsync=on and other stuff?

--
Alex Ignatov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company