Re: more numeric stuff

On Wed, Aug 4, 2010 at 4:07 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Robert Haas <robertmhaas(at)gmail(dot)com> writes:
>> I have a couple ideas for further work on the numeric code that I want
>> to get feedback on.
>
>> 1. Cramming it down some more.  I propose that we introduce a third
>> format with a one-byte header: 1 bit for sign, 3 bits for dynamic
>> scale, and 4 bits for weight (the first of which is a sign bit).  This
>> might seem crazy,
>
> Yes, it does.  In the first place it isn't going to work conveniently
> because NumericDigit requires int16 alignment.

It is definitely not convenient. I'm not disputing that.

> In the second, shaving
> just one byte doesn't seem like enough win to be worth the trouble.
> I don't believe your "billion rows" argument because you aren't
> factoring in the result of row-level alignment padding --- most of the
> time you're not going to win anything.

Row-level alignment padding is a problem, and on very short rows, or
rows where numeric is the only varlena, you may see no benefit. But
if there are multiple text or numeric columns packed up next to each
other, things are more promising.

>> We don't need any special
>> marker to indicate that the 1-byte format is in use, because we can
>> deduce it from the length of the varlena (after excluding the header):
>> even = 2b or 4b header, odd = 1b header.  There can't be any
>> odd-length numerics already on disk, so there shouldn't be any
>> compatibility break for pg_upgrade to worry about.
>
> Really?  Not sure this is true, because numerics can be toast-compressed.
> It hardly ever happens, but to do this that's not good enough.

I was thinking of it like PG_GETARG_TEXT_PP() and similar - we would
detoast compressed and external datums, but leave packed ones as-is.
At that point you should have an accurate length count, and can decide
what to do.

>> 2. Don't untoast/don't copy.
>
> This would be good, but I'm not sure how to do it.  The main problem
> again is NumericDigit alignment.  Only about half the time is the digit
> array going to be aligned the way you need, so that puts a real crimp
> in the possible win.  (In fact, if we assume the previous field is more
> than byte aligned and the toast header is one byte, then the digit array
> is *never* properly aligned on disk :-()

This is another reason why I think a 1-byte numeric header would be
good to have.

> One possibility is to have an additional toasting rule that forces
> odd-byte-alignment of a field's one-byte header.  But it's a bit hard to
> argue that numeric deserves the additional overhead that that would put
> into all the core tuple forming/deforming logic.

Yeah, plus we'd be adding more alignment padding for an extremely
tenuous performance gain. The benchmarks I did this morning seem to
indicate that the extra palloc/pfree/memcpy overhead is only barely
more than zero, so it only makes sense if we can get it without
suffering other penalties.

>> 3. 64-bit arithmetic.  Right now, mul_var() and div_var() use int for
>> arithmetic, but haven't we given up on supporting platforms without
>> long long?  I'm not sure I'm motivated enough to write the patch
>> myself, but it seems like 64-bit arithmetic would give us a lot more
>> room to postpone carries.
>
> I don't think this would win unless we went to 32-bit NumericDigit,
> which is a problem from the on-disk-compatibility standpoint,

This would increase the average size of a Numeric value considerably,
so it would be a very BAD thing IMO.

> not to
> mention making the alignment issues even worse.  Postponing carries is
> good, but we have enough headroom for that already --- I really doubt
> that making the array elements wider would save anything noticeable
> unless you increase NBASE.

I dunno, it was just a thought, based on some quick benchmarking that
indicated some possible hotspots in that area. But I didn't test it
carefully enough to be sure.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise Postgres Company