Re: Crash report for some ICU-52 (debian8) COLLATE and work_mem values

Peter Geoghegan wrote:

> I'd be interested to see if anything changed when -1 was passed as
> both sourceLength and targetLength to ucol_strcollUTF8(). You'd have
> to build Postgres yourself to test this, but it would just work, since
> we don't actually avoid NUL termination, even though in principled we
> could with ICU.

It doesn't seem to change the outcome:

*** varlena.c~ 2017-07-10 22:26:20.000000000 +0200
--- varlena.c 2017-08-02 12:57:25.997265001 +0200
***************
*** 2137,2144 ****

status = U_ZERO_ERROR;
result =
ucol_strcollUTF8(sss->locale->info.icu.ucol,
!
a1p, len1,
!
a2p, len2,

&status);
if (U_FAILURE(status))
ereport(ERROR,
--- 2137,2144 ----

status = U_ZERO_ERROR;
result =
ucol_strcollUTF8(sss->locale->info.icu.ucol,
!
a1p, -1, /* len1 */
!
a2p, -1, /*len2,*/

&status);
if (U_FAILURE(status))
ereport(ERROR,

For the case when its ends up crashing in memcpy, the backtrace now looks
like this:

#0 __memcpy_sse2_unaligned ()
at ../sysdeps/x86_64/multiarch/memcpy-sse2-unaligned.S:35
#1 0x00007f0156812e6b in memcpy (__len=8589934592, __src=0x7efbeac89010,
__dest=<optimized out>) at
/usr/include/x86_64-linux-gnu/bits/string3.h:51
#2 ucol_CEBuf_Expand (ci=<optimized out>, status=0x7ffd49c4ca98,
b=0x7ffd49c4ab60) at ucol.cpp:7009
#3 UCOL_CEBUF_PUT (status=0x7ffd49c4ca98, ci=0x7ffd49c4bde0, ce=1493173509,
b=0x7ffd49c4ab60) at ucol.cpp:7022
#4 ucol_strcollRegular (sColl=sColl(at)entry=0x7ffd49c4bde0,
tColl=tColl(at)entry=0x7ffd49c4bf90, status=status(at)entry=0x7ffd49c4ca98)
at ucol.cpp:7163
#5 0x00007f0156818177 in ucol_strcollRegularUTF8 (coll=0x2286100,
source=source(at)entry=0x3651829 "쳭喩zz",
sourceLength=sourceLength(at)entry=-1, target=<optimized out>,
targetLength=targetLength(at)entry=-1, status=status(at)entry=0x7ffd49c4ca98)
at ucol.cpp:8023
#6 0x00007f0156819d36 in ucol_strcollUseLatin1UTF8 (status=<optimized out>,
tLen=<optimized out>, target=<optimized out>, sLen=<optimized out>,
source=<optimized out>, coll=<optimized out>) at ucol.cpp:8108
#7 ucol_strcollUTF8_52 (coll=<optimized out>,
source=source(at)entry=0x3651829 "쳭喩zz", sourceLength=<optimized out>,
sourceLength(at)entry=-1, target=<optimized out>,
target(at)entry=0x3651799 "콗喩zx㎍", targetLength=targetLength(at)entry=-1,
status=status(at)entry=0x7ffd49c4ca98) at ucol.cpp:8770
#8 0x00000000007cc7df in varstrfastcmp_locale (x=56956968, y=56956824,
ssup=<optimized out>) at varlena.c:2139
#9 0x0000000000817168 in ApplySortAbbrevFullComparator (ssup=0x2280338,
isNull2=<optimized out>, datum2=<optimized out>, isNull1=<optimized out>,
datum1=<optimized out>) at
../../../../src/include/utils/sortsupport.h:263
#10 comparetup_datum (a=0x7f0141c14228, b=0x7f0141c14240,
state=<optimized out>) at tuplesort.c:4350
#11 0x0000000000815b1f in qsort_tuple (a=0x7f0141c14210, n=<optimized out>,
n(at)entry=106, cmp_tuple=cmp_tuple(at)entry=0x817060 <comparetup_datum>,
state=state(at)entry=0x2280128) at qsort_tuple.c:104
.....

Anyway I'll try to build ICU-52 without optimization to avoid these
<optimized out> parameters.

Also running the same tests with a self-compiled ICU-59 versus debian's
packaged
ICU-52 results in *no crash at all*. So maybe the root cause of these crashes
is
that a critical fix in ICU has not been backported?

I note that the list of ICU locales available in pg_collation is different
though.
When compiled with ICU-59, initdb produces "only" 581 collname
matching '%icu%', versus 1741 with ICU-52. The ICU version
is the only thing that differs between both cases.
But anyway in the list of collations that crash for me with ICU-52, a fair
number
(~50) are still there and work in ICU-59, so I think the comparison is still
meaningful.

Best regards,
--
Daniel Vérité
PostgreSQL-powered mailer: http://www.manitou-mail.org
Twitter: @DanielVerite