Re: pg_dump/pg_restore problems with 7.4.1

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> Ben Marklein wrote:
>> db=# CREATE UNIQUE INDEX person_info_username_ix ON
>> person_info USING btree (username);
>> ERROR: could not create unique index
>> DETAIL: Table contains duplicated values.
>>
>> Of course, this index existed in the 7.2 DB, so it
>> seems odd that it should not be possible to recreate.

> Not sure. Never heard of this before.

It seems odd to me too. Is it possible that the new database uses a
different character set encoding and/or different locale, such that
two strings previously considered unequal are now equal? (Of course,
you'd think the count(*) check would find any such cases...)

>> 2) While importing I get: "ERROR: invalid memory
>> alloc request size 1073741824". I tracked this down
>> to a line with some non-ASCII characters.

> Sounds like the same problem, but I don't remember the solution.

I think this is this problem:

2004-01-17 21:15 tgl

* src/backend/commands/: copy.c (REL7_4_STABLE), copy.c: Don't use
%s-with-precision format spec to truncate data being displayed in a
COPY error message. It seems that glibc gets indigestion if it is
asked to truncate strings that contain invalid UTF-8 encoding
sequences. vsnprintf will return -1 in such cases, leading to
looping and eventual memory overflow in elog.c. Instead use our
own, more robust pg_mbcliplen routine. I believe this problem
accounts for several recent reports of unexpected 'out of memory'
errors during COPY IN.

Until we get around to issuing a 7.4.2, the easiest solution is to fix
the data to match the database encoding (or vice versa). You'd actually
have to do that anyway --- the bug is merely that you get a useless
error message instead of a helpful one when the incoming data is bogus
according to the database encoding.

regards, tom lane