Re: What is the maximum encoding-conversion growth rate, anyway?

From: Bruce Momjian <bruce(at)momjian(dot)us>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Tatsuo Ishii <t-ishii(at)sra(dot)co(dot)jp>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: What is the maximum encoding-conversion growth rate, anyway?
Date: 2007-07-17 00:59:33
Message-ID: 200707170059.l6H0xX305870@momjian.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers


Where are we on this?

---------------------------------------------------------------------------

Tom Lane wrote:
> I just rearranged the code in mbutils.c a little bit to make it more
> robust if conversion of an over-length string is attempted, and noted
> this comment:
>
> /*
> * When converting strings between different encodings, we assume that space
> * for converted result is 4-to-1 growth in the worst case. The rate for
> * currently supported encoding pairs are within 3 (SJIS JIS X0201 half width
> * kanna -> UTF8 is the worst case). So "4" should be enough for the moment.
> *
> * Note that this is not the same as the maximum character width in any
> * particular encoding.
> */
> #define MAX_CONVERSION_GROWTH 4
>
> It strikes me that this is overly pessimistic, since we do not support
> 5- or 6-byte UTF8 characters, and AFAICS there are no 1-byte characters
> in any supported encoding that require 4 bytes in another. Could we
> reduce the multiplier to 3? Or even 2? This has a direct impact on the
> longest COPY lines we can support, so I'd like it not to be larger than
> necessary.
>
> regards, tom lane
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Have you searched our list archives?
>
> http://archives.postgresql.org

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Bruce Momjian 2007-07-17 01:34:28 Re: [COMMITTERS] pgsql: Create hooks to let a loadable plugin monitor (or even replace)
Previous Message Gregory Stark 2007-07-16 21:37:57 Re: compiler warnings on the buildfarm