Re: BUG #15307: Low numerical precision of (Co-) Variance

From: Madeleine Thompson <madeleineth(at)gmail(dot)com>
To: pgsql-hackers(at)lists(dot)postgresql(dot)org
Cc: Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com>
Subject: Re: BUG #15307: Low numerical precision of (Co-) Variance
Date: 2018-09-27 05:12:23
Message-ID: 153802514313.25845.6696084477831363882.pgcf@coridan.postgresql.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs pgsql-hackers

The following review has been posted through the commitfest application:
make installcheck-world: tested, passed
Implements feature: tested, passed
Spec compliant: not tested
Documentation: not tested

This is my first PostgreSQL review. Hopefully I'm getting it right. I independently ran into the bug in question and found that the author had already written a patch. I'm happy the author wrote this.

(1) I am not experienced with PostgreSQL internals, so I can't comment on the coding style or usage of internal functions.

(2) The patch applies cleanly to ce4887bd025b95c7b455fefd817a418844c6aad3. "make check", "make check-world", and "make install" pass.

(3) The patch addresses the numeric inaccuracy reported in the bug. (Yay!) I believe the tests are sufficient to confirm that it works as intended. I don't think the test coverage *before* this patch was sufficient, and I appreciate the improved test coverage of the combine functions. I verified the new tests manually.

(4) The comments cite Youngs & Cramer (1971). This is a dated citation. It justifies its algorithms based on pre-IEEE754 single-precision arithmetic. Most notably, it assumes that floating-point division is not exact. That said, the algorithm is still a good choice; it is justified now because compared to the standard Welford variance, it is less prone to causing pipeline stalls on a modern CPU. Maybe link to Schubert & Gentz 2018 (https://dl.acm.org/citation.cfm?id=3223036) instead. The new float8_combine combine code is (almost) S&G eqn. 22; the float8_accum code is S&G eqn. 28. float8_regr_accum and float8_regr_combine are S&G eqn. 22.

(5) I think the comment /* Watch out for roundoff error producing a negative variance */ and associated check are obsolete now.

The new status of this patch is: Waiting on Author

In response to

Responses

Browse pgsql-bugs by date

  From Date Subject
Next Message Madeleine Thompson 2018-09-27 05:46:25 Re: BUG #15307: Low numerical precision of (Co-) Variance
Previous Message Michael Paquier 2018-09-27 04:44:30 Re: BUG #15402: Hot standby server with archive_mode=on keeps initial WAL segments

Browse pgsql-hackers by date

  From Date Subject
Next Message Madeleine Thompson 2018-09-27 05:46:25 Re: BUG #15307: Low numerical precision of (Co-) Variance
Previous Message Michael Paquier 2018-09-27 05:10:59 Re: Let's stop with the retail rebuilds of src/port/ files already