Re: [HACKERS] interesting observatation regarding views and V7.0

Tom Lane wrote:

> Don Baccus <dhogaza(at)pacifier(dot)com> writes:
>
> > Also...interbase's "text" type is apparently compressed, and that's
> > an interesting idea for "text" itself (as opposed to "varchar()" of
> > a given size). Someone who just says "text" probably wants to be
> > able to stuff as much text into the column as possible, I know
> > I do!
>
> Just quietly make text compressed-under-the-hood, you mean? Hmm.
> Interesting idea, all right, and it wouldn't create any long-term
> compatibility problem since users couldn't see it directly. ...

If we wheren't in BETA code freeze right now, I'd call for
another month delay - surely.

> > The price of compression/decompression is to some extent
> > balanced by not having to drag as many bytes around during joins
> > and sorts and the like.
>
> Also, there could be a threshold: don't bother trying to compress
> fields that are less than, say, 1K bytes.
>
> Jan, what do you think? I might be able to find some time to try this,
> if you approve of the idea but just don't have cycles to spare.

It's a very temping solution, turn "text" into "lztext"
silently, and revert that internal changes in the next
release again while implementing TOAST. Remember that the
lztext I implemented had the mentioned threshold paramenter -
say 256 - from the very beginning. And you know 256->1K is a
one-liner in my coding style. Moreover, it was a global
parameter set driven value, and thus potentially prepared to
be a runtime configurable one (the other values of the
parameter set where minimum compression ratio to gain,
maximum result size to force compression even if ratio below,
GOOD size to stop history lookup and finally history lookup
GOOD lowering factor during lookups).

The algorithm I used for compression is one, loosing possible
compression ratio to gain speed. It uses a poor XOR
combination of the next 4 input-bytes, to lookup a history
table - and that's anything but perfect from a hashing
algorithms point of view. But it was enough to make a 50+
column view fit easily into pg_rewrite. And that's what it
was made for.

Anyway, there are far too many direct references to VARDATA
on "text" plus all the assumptions on binary compatibility
between text, varchar etc. in the code, to start on it during
BETA.

Thus, I see a good chance for a 7.1 release, really soon
after 7.0. Then have a longer delay for the next one,
featuring TOAST.

Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#========================================= wieck(at)debis(dot)com (Jan Wieck) #