Re: Stock Market Price Data & postgreSQL? HELLPPP Please

On Wed, Aug 19, 2009 at 4:22 PM, Sanjay Arora<sanjay(dot)k(dot)arora(at)gmail(dot)com> wrote:
> Hello All
>
> I want to store stock market price data in postgreSQL and some of my
> associates are saying this is not workable, not only in postgreSQL but
> in any database. Arguments given are listed below. Kindly comment and
> if possible suggest solutions and pointers to further reading.
>
> - This is Time Series Data (I don't know what that is except that it
> relates to data marked/related to time) and not suited to a RDBMS.
> - You need it in Esper (a CEP engine used by Marketcetera, an open
> source trading platform) which needs to get a data stream, whereas an
> RDBMS will return a recordset.
> - A RDBMS including postgreSQL simply cannot handle the amount of
> real-time data coming in from a stock exchange.
> - A RDBMS simply cannot provide data, even if you build streams from
> recordsets returned, at the speeds required by any technical analysis
> charting program.
>
> My level of knowledge is basic SysAdmin & Programming Concepts, not
> actual programming. I will be getting someone to build a solution for
> me. Need the correct direction please. Again, shoestring budget, all
> open source pointers please, unless they are for reading & general
> knowledge.
>
> I am hoping for an adapted postgreSQL solution, can't spend more than
> few hundred dollars for this....so please...HELLLPPP ;-)

I think your first step is to stand back, think about the problem as a
whole and see if you can't define your requirements better:

*) how much data comes in each day? how much rolls out?
*) what types of analysis of the data would you like to do?
*) what are your performance expectations?

For example, try and describe what you would like to do in detail.
The data you would import, how you would like it structured, and a
typical query. While the amount of price data a stock market produces
is immense, these types of problems tend to scale very well
horizontally. So let's define the basic amount of work you'd like to
do, then build off that.

merlin