Re: [HACKERS] "24" < INT_MIN returns TRUE ???

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Bruce Momjian <maillist(at)candle(dot)pha(dot)pa(dot)us>
Cc: Jan Wieck <jwieck(at)debis(dot)com>, PostgreSQL HACKERS <pgsql-hackers(at)postgreSQL(dot)org>
Subject: Re: [HACKERS] "24" < INT_MIN returns TRUE ???
Date: 1999-07-09 14:49:36
Message-ID: 2336.931531776@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

>> Inside of pg_atoi(), the value is read into a long. Comparing
>> a small positive long like 24 against INT_MIN returns TRUE -
>> dunno how. Putting INT_MIN into another long variable and
>> comparing the two returns the expected FALSE - so what's
>> going on here? long, int32 and int have all 4 bytes here.

I believe the problem is that the compiler is deciding that INT_MIN
is of type "unsigned int" or "unsigned long", whereupon the type
promotion rules will cause the comparison to be done in
unsigned-long arithmetic. And indeed, 24 < 0x80000000 in unsigned
arithmetic. When you compare two long variables,
you get the desired behavior of signed long comparison.

Do you have <limits.h>, and if so how does it define INT_MIN?

The default INT_MIN provided at the top of numutils.c is clearly
prone to cause this problem:
#ifndef INT_MIN
#define INT_MIN (-0x80000000L)
#endif
If long is 32 bits then the constant 0x80000000L will be classified
as unsigned long by an ANSI-compliant compiler, whereupon the test
in pg_atoi fails.

The two systems I have here both avoid this problem in <limits.h>,
but I wonder whether you guys have different or no <limits.h>.

I recommend a two-pronged approach to dealing with this bug:

1. The default INT_MIN ought to read
#define INT_MIN (-INT_MAX-1)
so that it is classified as a signed rather than unsigned long.
(This is how both of my systems define it in <limits.h>.)

2. The two tests in pg_atoi ought to read
if (l < (long) INT_MIN)
...
if (l > (long) INT_MAX)

The second change should ensure we get a signed-long comparison
even if <limits.h> has provided a broken definition of INT_MIN.
The first change is not necessary to fix this particular bug
if we also apply the second change, but I think leaving INT_MIN
the way it is is trouble waiting to happen.

Bruce, I cannot check this since my system won't show the failure;
would you un-reverse-out the prior patch, add these changes, and
see if it works for you?

regards, tom lane

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 1999-07-09 15:02:12 Re: [HACKERS] "24" < INT_MIN returns TRUE ???
Previous Message Mark Hollomon 1999-07-09 14:46:51 Re: [HACKERS] Hashing passwords (was Updated TODO list)