Re: [HACKERS] Invalid unicode in COPY problem

John Hansen wrote:
> Tatsuo Ishii wrote:
>
>>We have developed patches which relaxes the character
>>validation so that PostgreSQL accepts invalid characters. It
>>works like this:
>
>
> That is just plain 100% wrong!!
> Under no circumstances should there be invalid data in a database.
> And if you're trying to make a database of invalid data, then at
> least encode it using a valid encoding.
>
> In fact, I've proposed strengthening the validation routines for UTF-8.
>
> ... John
>

Under most circumstances I would agree with you completely. In my
case though I have to decide between risking a loss of a user's data or
attempt to store the file name in some manner that would return the same
name used by the file system.

The user (or one of his/her users in the case of an admin) may be
completely unaware of the file name being an invalid unicode name. The
file itself though may still be quite valid and contain information
worthy of backing up. I could notify the user/admin that the name is not
valid but there is no way I could rely on the name being changed. Given
the choices, I would prefer to attempt to store/use the file name with
the invalid unicode character than simply ignore the file.

Is there a way to store the name in raw binary? If so, would this not
be safe because to postgresql it should no longer matter what data is or
represents, right? Maybe there is a third option I am not yet concidering?

Madison

--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Madison Kelly (Digimer)
TLE-BU, The Linux Experience; Back Up
http://tle-bu.thelinuxexperience.com
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-