Re: [WIP]Vertical Clustered Index (columnar store extension) - take2

08.04.2025 13:29, Aya Iwata (Fujitsu) wrote:
> Hi Yura san,
>
>
>> I just don't get, why it should be "in-memory"? All the same things you
>> describe further, but storing in paged index on-disk with caching
>> through shared_buffers - why this way it wouldn't work?
>
> We make the columnar store resident in memory for maximum search performance.
> But I'm not very particular about this. Comments are welcome.

I just wanted to say: there is no need to be super fast.
There is the need to be remarkably faster than it is now.

ClickHouse, DuckDB, Vertica - they are not in-memory, they are disk based.
But they are very fast.
If PostgreSQL will be just as twice slower as ClickHouse, it will be very
great! Most of users will not setup ClickHouse at all then, because twice
slower is still very fast.

Databases could be very huge. Even when they are in "columnar" format,
which usually consumes less space. And memory is still costs more than disk
space.

Certainly there are users who think they need "in-memory". But the truth is
very few of them really need "in-memory".

All of this is just my opinion. I could be wrong.

>> -----Original Message-----
>> From: Yura Sokolov <y(dot)sokolov(at)postgrespro(dot)ru>
>> Sent: Wednesday, January 15, 2025 11:44 PM
>> To: pgsql-hackers(at)lists(dot)postgresql(dot)org
>> Subject: Re: [WIP]Vertical Clustered Index (columnar store extension) - take2
>>
>> 07.10.2024 17:53, Aya Iwata (Fujitsu) wrote:
>>> Hi All,
>>>
>>> Suggestions
>>>
>>> ==========
>>>
>>> When analyzing real-time data collected by PostgreSQL,
>>>
>>> it can be difficult to tune the current PostgreSQL server for
>>> satisfactory performance.
>>>
>>> Therefore, we propose Vertical Clustered Indexing (VCI), an in-memory
>>> column store function that holds data in a state suitable for business
>>> analysis and is also expected to improve analysis performance.
>>
>> I just don't get, why it should be "in-memory"? All the same things you
>> describe further, but storing in paged index on-disk with caching
>> through shared_buffers - why this way it wouldn't work?

--
regards
Yura Sokolov aka funny-falcon