Re: Querying sporadic time series type data.

From: Tim Uckun <timuckun(at)gmail(dot)com>
To: Steven Pousty <steve(dot)pousty(at)gmail(dot)com>
Cc: "David G(dot) Johnston" <david(dot)g(dot)johnston(at)gmail(dot)com>, pgsql-sql <pgsql-sql(at)lists(dot)postgresql(dot)org>
Subject: Re: Querying sporadic time series type data.
Date: 2020-10-04 08:29:24
Message-ID: CAGuHJrMd790Fa0GeSfCO5eOj8ePp+LtGKpgSrg4jm5anbVjF=g@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-sql

> select... from table where [9,xdate::date]::daterange @:> datecolumn and metric = 'measure you want' order by date desc limit 1

In the correlated subquery a simple select date, value from metrics
where metric_id=x order by datetime desc limit 1 would get you the
figure you want. but of course this has to be repeated for every
metric you collect and if you wanted to do something fancy like plot
some measure over a year based on monthly figures it's massively
inefficient.

> Back to the design of your db, it sounds like you have a fixed set of metrics you are recording. If this is so I think making them distinct columns in your data set is preferable than the scheme you are using. But if you are going to be adding many new metrics frequently then your design makes sense to me.

The problem with this is that most of the rows will only have one
column filled in. In fact probably all of them are.

I am starting to think the best way to handle this is by creating some
time slot tables with all the columns and populating with batch jobs.
A daily table, weekly table, monthly table etc. Another option might
be to go ahead and write that slow and io heavy query and save it as a
materialized view.

In response to

Browse pgsql-sql by date

  From Date Subject
Next Message Francesco De Angelis 2020-10-04 14:40:54 parallelisation of queries
Previous Message Steven Pousty 2020-10-04 00:42:14 Re: Querying sporadic time series type data.