Re: Zstandard support for toast compression

On Mon, May 23, 2022 at 12:33 AM Michael Paquier <michael(at)paquier(dot)xyz> wrote:
> On Thu, May 19, 2022 at 04:12:01PM -0400, Robert Haas wrote:
> > On Thu, May 19, 2022 at 4:20 AM Michael Paquier <michael(at)paquier(dot)xyz> wrote:
> >> Btw, shouldn't we have something a bit more, err, extensible for the
> >> design of an extensible varlena header? If we keep it down to some
> >> bitwise information, we'd be fine for a long time but it would be
> >> annoying to review again an extended design if we need to extend it
> >> with more data.
> >
> > What do you have in mind?
>
> A per-varlena checksum was one thing that came into my mind.

It's a bit hard for me to believe that such a thing would be
desirable. I think it makes more sense to checksum blocks than datums,
because:

(1) There might be a lot of really small datums, and storing checksums
for all of them could be costly, or
(2) The datums could on the other hand be really big, and then the
checksum is pretty non-specific about where the problem has happened.

YMMV, of course.

--
Robert Haas
EDB: http://www.enterprisedb.com