Re: [HACKERS][Proposal] LZ4 Compressed Storage Manager

čt 11. 4. 2019 v 18:18 odesílatel Bruce Momjian <bruce(at)momjian(dot)us> napsal:

> On Sun, Mar 31, 2019 at 05:25:51PM +0300, Николай Петров wrote:
> > Why it should be implemented on Storage Manager level instead of usage
> > Pluggable storage API [9]?
> > - From my perspective view Storage Manager level implementation
> > allows to focus on proper I/O operations and compression.
> > It allows to write much more simple realization. It's because of
> > Pluggable storage API force you to implement more complex
> > interfaces. To be honest, I am really hesitating about this point,
> > especially because of Pluggable storage API allows to create
> > extension without core code modification and it potentially allows
> > to use more perfective compression algorithms (Table Access Manager
> > allows you to get more information about storing data).
> >
> > I would like to implement a proof of concept
> > and have a couple of questions:
> > - your opinion about necessity of this feature
> > (Compressed Storage Manager)
> > - Is it good idea to implement DB compression on Storage Manager
> > level? Perhaps it is better to use Pluggable storage API.
> > - Is there any reason to refuse this proposal?
> > - Are there any circumstances what didn't allow to implement
> > Compressed Storage Manager?
>
> Stepping back a bit, there are several levels of compression:
>
> 1. single field
> 2. across all fields in a row
> 3. across rows on a single page
> 4. across all rows in a table
> 5. across tables in a database
>

> We currently do #1 with TOAST, and your approach would allow the first
> three. #4 feels like it is getting near the features of columnar
> storage. I think it is unclear if adding #2 and #3 produce enough of a
> benefit to warrant special storage, given the complexity and overhead of
> implementing it.
>

@4 compression over columns on page are probably much more effective. But
there can some preprocessing stage, where rows can be transformed to
columns.

This doesn't need real column store, and can helps lot of. Real column
store has sense when columns are separated to different pages. But for
compressions, we can transform rows to columns without real column storage.

Probably 8kB page is too small for this case.

Regards

Pavel

> I do think the Pluggable storage API is the right approach, and, if you
> are going to go that route, adding #4 compression seems very worthwhile.
>
> --
> Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
> EnterpriseDB http://enterprisedb.com
>
> + As you are, so once was I. As I am, so you will be. +
> + Ancient Roman grave inscription +
>
>
>