Re: [HACKERS] Custom compression methods

From: Justin Pryzby <pryzby(at)telsasoft(dot)com>
To: Dilip Kumar <dilipbalaut(at)gmail(dot)com>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>, Alexander Korotkov <a(dot)korotkov(at)postgrespro(dot)ru>, David Steele <david(at)pgmasters(dot)net>, Ildus Kurbangaliev <i(dot)kurbangaliev(at)gmail(dot)com>, Dmitry Dolgov <9erthalion6(at)gmail(dot)com>, pgsql-hackers(at)lists(dot)postgresql(dot)org
Subject: Re: [HACKERS] Custom compression methods
Date: 2021-02-28 04:18:21
Message-ID: 20210228041821.GB20769@telsasoft.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On my PC, this new change is causing a test failure:

SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
- substr
-----------------------------------------------------
- 01234567890123456789012345678901234567890123456789
-(1 row)
-
+ERROR: compressed lz4 data is corrupt

@@ -119,15 +119,15 @@ lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
int32 rawsize;
struct varlena *result;

- /* allocate memory for holding the uncompressed data */
- result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);

- /* decompress partial data using lz4 routine */
+ /* decompress the data */
rawsize = LZ4_decompress_safe_partial((char *) value + VARHDRSZ_COMPRESS,
VARDATA(result),
VARSIZE(value) - VARHDRSZ_COMPRESS,
slicelength,
- VARRAWSIZE_4B_C(value));
+ slicelength);

Also, in the tests, you have this at both the top and bottom of the file:

src/test/regress/sql/compression.sql:\set HIDE_COMPRESSAM false
src/test/regress/sql/compression.sql:\set HIDE_COMPRESSAM false

Whereas the patch I sent had at the end:

+\set HIDE_COMPRESSAM on

("on" is the default when run under pg_regress)

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message japin 2021-02-28 06:36:01 Re: NOT VALID for Unique Indexes
Previous Message Joel Jacobson 2021-02-28 03:58:05 Re: regexp_positions()