Re: Zstandard support for toast compression

Greetings,

* Tom Lane (tgl(at)sss(dot)pgh(dot)pa(dot)us) wrote:
> Stephen Frost <sfrost(at)snowman(dot)net> writes:
> > * Robert Haas (robertmhaas(at)gmail(dot)com) wrote:
> >> Yeah - I think we had better reserve the fourth bit pattern for
> >> something extensible e.g. another byte or several to specify the
> >> actual method, so that we don't have a hard limit of 4 methods. But
> >> even with such a system, the first 3 methods will always and forever
> >> be privileged over all others, so we'd better not make the mistake of
> >> adding something silly as our third algorithm.
>
> > In such a situation, would they really end up being properly distinct
> > when it comes to what our users see..? I wouldn't really think so.
>
> It should be transparent to users, sure, but the point is that the
> first three methods will have a storage space advantage over others.
> Plus we'd have to do some actual work to create that extension mechanism.
>
> I'm with Robert in that I do not see any urgency to add another method.
> The fact that Stephen is already questioning whether LZ4 should have
> been added first is not making me any more eager to jump here.
> Compression methods come, and they go, and we do not serve anyone's
> interest by being early adopters.

I'm getting a bit of deja-vu here from when I was first trying to add
TRUNCATE as a GRANT'able option and being told we didn't want to burn
those precious bits.

But, fine, then I'd suggest to Michael that he work on actively solving
the problem we've now got where we have such a limited number of bits,
and then come back and add Zstd after that's done. I disagree that we
should be pushing back so hard on adding Zstd in general, but if we are
going to demand that we have a way to support more than these few
compression options before ever adding any new ones (considering how
long it's taken Zstd to get to the level it is now, we're talking
about close to a *decade* from such a new algorithm showing up and
getting to a similar level of adoption, and then apparently more because
we don't feel it's 'ready' yet), then let's work towards that and not
complain when it shows up that it's not needed yet (as I fear would
happen ... and just leave us unable to make useful progress).

Thanks,

Stephen