Re: Postgres as Historian

On Mon, Aug 2, 2010 at 3:20 AM, Hardik Belani <hardikbelani(at)gmail(dot)com> wrote:
> We are using postgres as RDBMS for our product. There is a requirement
> coming for a feature which will require me to store data about various data
> points (mainly numbers) on a time scale. Data measurement is being taken
> every few secs/mins based and it is needed to be stored for statistical
> analysis. Now this requires numbers (integers/floats) to be stored at every
> mins.
>
> For this i can create a table with number and time (may be time offset
> instead of timestamp) as columns. But still it will require me to store huge
> number of rows in the order of few millions. Data is read only and only
> inserts can happen. But I need to perform all kinds of aggregation to get
> various statistics. for example: daily avg, monthly avg etc..
>
> We already are using postgres for our product so using postgres does not add
> any additional installation requirement and hence it is a bit easier.
>
> Would you recommand postgres for this kind of requirement and will be
> provide the performance. OR do you recommand any other database meant
> for such requirements. I am also searching for a good historian database if
> postgres doesn't suppport.

You can certainly use Postgres in this kind of environment, and I
have. Of course, if the volume of data is higher than your hardware
can keep up with, then you're going to have problems. Disabling
synchronous_commit may help, if losing the last few transactions is
acceptable in the event of a system crash; appropriate use of table
partitioning may help, too. There are certainly databases out there
that are better optimized for this case (consider the rrd stuff built
into mrtg, for example), but they're also not as feature-rich, so it's
a trade-off.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise Postgres Company