Re: PostgreSQL, OLAP, and Large Clusters

On Thu, Sep 27, 2012 at 08:58:05AM +1200, Gavin Flower wrote:
> On 26/09/12 23:50, Ryan Kelly wrote:
> >Hi:
> >
> >The size of our database is growing rather rapidly. We're concerned
> >about how well Postgres will scale for OLAP-style queries over terabytes
> >of data. Googling around doesn't yield great results for vanilla
> >Postgres in this application, but generally links to other software like
> >Greenplum, Netezza, and Aster Data (some of which are based off of
> >Postgres). Too, there are solutions like Stado. But I'm concerned about
> >the amount of effort to use such solutions and what we would have to
> >give up feature-wise.
> >
> >We love a lot of the features that we currently have that (I think)
> >would only be available with Postgres: arrays, hstore, gin/gist indexes,
> >extensions. Not to mention a lot of the other great SQL standard
> >features not available in other open-source databases, especially window
> >functions and CTEs.
> >
> >Of course, migration to another solution is feasible (with enough
> >effort, of course), but given my experiences with Postgres and the
> >support provided by the community that is second to none, I'd very much
> >like to stay with PostgreSQL.
> >
> >Thoughts?
> >
> >-Ryan Kelly
> >
> >
> You might find it worth while to consult a reputable vendor of
> Postgres, such as EnterpriseDB, who contribute change back to the
> community. As you may have rather specialized needs beyond the
> capacity of these mailing lists. Also, I suspect paying them to
> help improve Postgres's capability to handle such large datasets
> would be cheaper than an Oracle Licence and costs of migration!
Contacting EnterpriseDB has been on my list of things to do for a while
now. I should probably get around to do doing that :)

> I don't have any specific knowledge of the current ability of
> Postgres to support such high loads, so I can't comment meaningfully
> on that. And even if I could, and it was favourable, I would still
> recommend getting a competent company to provide consultancy for
> your particular situation.
>
>
> Cheers,
> Gavin

Thanks,
-Ryan Kelly