Re: Postgres as In-Memory Database?

Hey Stefan,

@Hadi: Can you say something about usage of cstore FDW in-memory?
>
>
We designed cstore_fdw with the applications in mind where volume of data
is much larger than main memory. In general, columnar stores usually bring
two benefits:

1. Doing less disk I/O than row stores. We can skip reading entire columns
or column blocks that are not related to the given query. This is effective
when (a) volume of data is larger than main memory so OS cannot cache whole
dataset, (b) most of our queries only require a small subset of columns to
complete.

2. Vector processing and making better use of CPU. This usually helps most
when data is in memory. If data is in disk and is not cached, I/O cost is
usually higher than CPU cost, and vector processing may not help much.

cstore_fdw tries to optimize for #1. Also note that because we use
compression, more data can be cached in memory and chance of hitting disk
decreases.

But we don't do vector processing yet, and it is not our three month
timeline.

If you want to be able use more CPU cores in PostgreSQL, you can have a
look at CitusDB [1] which is built upon PostgreSQL and distributes queries
to use all cpu cores in a single or more machines.

[1] http://citusdata.com/

-- Hadi