In-Memory Columnar Store

From: knizhnik <knizhnik(at)garret(dot)ru>
To: pgsql-hackers(at)postgresql(dot)org
Cc: Abhijit Menon-Sen <ams(at)2ndQuadrant(dot)com>, Oleg Bartunov <obartunov(at)gmail(dot)com>
Subject: In-Memory Columnar Store
Date: 2013-12-09 19:40:41
Message-ID: 52A61CB9.3080906@garret.ru
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hello!

I want to annouce my implementation of In-Memory Columnar Store
extension for PostgreSQL:

Documentation: http://www.garret.ru/imcs/user_guide.html
Sources: http://www.garret.ru/imcs-1.01.tar.gz

Any feedbacks, bug reports and suggestions are welcome.

Vertical representation of data is stored in PostgreSQL shared memory.
This is why it is important to be able to utilize all available physical
memory.
Now servers with Tb or more RAM are not something exotic, especially in
financial world.
But there is limitation in Linux with standard 4kb pages for maximal
size of mapped memory segment: 256Gb.
It is possible to overcome this limitation either by creating multiple
segments - but it requires too much changes in PostgreSQL memory manager.
Or just set MAP_HUGETLB flag (assuming that huge pages were allocated in
the system).

I found several messages related with MAP_HUGETLB flag, the most recent
one was from 21 of November:
http://www.postgresql.org/message-id/20131125032920.GA23793@toroid.org

I wonder what is the current status of this patch?

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Noah Misch 2013-12-09 19:40:56 Re: UNNEST with multiple args, and TABLE with multiple funcs
Previous Message Robert Haas 2013-12-09 19:26:52 Re: shared memory message queues