about numerical accuracy of statistical functions

From: Yuanyuan Tian <ytian(at)umich(dot)edu>
To: pgsql-general(at)postgresql(dot)org
Subject: about numerical accuracy of statistical functions
Date: 2012-03-23 22:31:17
Message-ID: CALndwgELDid0-Y+VyvWAG_fznvZG9DSp57UdwteaQoQWMHAVxQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Hi,

I want to use statistical functions, such as variance(), stddev() and
covar_samp(), to compute some basic statistics of my data. I have
strict requirement for the numerical accuracy of the results. So, I
want to know whether the implementation of these functions are
numerically stable. For example, variance can be computed as
sum(x^2)/n - sum(x)^2/n^2. But this computation can lead to serious
accuracy problems with double precision. Can anybody shed lights on
what algorithms are used to implement these functions and whether they
are numerically stable?

Cheers,
Yuanyuan

Browse pgsql-general by date

  From Date Subject
Next Message Tom Lane 2012-03-24 00:34:52 Re: Change the default [tgenabled] for new "internal" triggers ?
Previous Message Adrian Klaver 2012-03-23 19:35:16 Re: PostgreSQL 64 Bit XIDs - Transaction IDs