From: | Robert Burgholzer <rburghol(at)vt(dot)edu> |
---|---|
To: | pgsql-performance(at)postgresql(dot)org |
Subject: | Optimizing Time Series Access |
Date: | 2014-04-08 21:20:19 |
Message-ID: | CACT-NGJAMP0KULGMf3B8+0YqeywD1i34R7Okzb74Gbzn7_oU-A@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-performance |
I am looking for advice on dealing with large tables of environmental model
data and looking for alternatives to my current optimization approaches.
Basically, I have about 1 Billion records stored in a table which I access
in groups of roughly 23 Million at a time. Which means that I have
somewhere in the neighborhood of 400-500 sets of 23Mil points.
The 23Mil that I pull at a time are keyed on 3 different columns, it's all
indexed, and retrieval happens in say, 2-3 minutes (my hardware is so-so).
So, my thought is to use some kind of caching and wonder if I can get
advice - here are my thoughts on options, would love to hear others:
* use cached tables for this - since my # of actual data groups is small,
why not just retrieve them once, then keep them around in a specially named
table (I do this with some other stuff, using a 30 day cache expiration)
* Use some sort of stored procedure? I don't even know if such a thing
really exists in PG and how it works.
* Use table partitioning?
Thanks,
/r/b
--
--
Robert W. Burgholzer
'Making the simple complicated is commonplace; making the complicated
simple, awesomely simple, that's creativity.' - Charles Mingus
Athletics: http://athleticalgorithm.wordpress.com/
Science: http://robertwb.wordpress.com/
Wine: http://reesvineyard.wordpress.com/
From | Date | Subject | |
---|---|---|---|
Next Message | Tom Lane | 2014-04-08 21:26:37 | Re: Performance regressions in PG 9.3 vs PG 9.0 |
Previous Message | Dhananjay Singh | 2014-04-08 20:40:09 | Re: Nested loop issue |