Re: Invalid byte sequence

ok, thanks a lot

On Thu, Jul 1, 2021 at 9:58 AM Holger Jakobs <holger(at)jakobs(dot)com> wrote:

> Or course you could delete all these lines, but this may lead to data loss.
>
> Therefore, if it's not just a singe line, I think it's best to solve the
> problem from its root. So check how the file was generated and correct it
> there.
>
> But this is beyond PostgreSQL.
>
>
> Am 01.07.21 um 09:56 schrieb Yambu:
>
> There are a number of such lines, the problem is in identifying all of
> them and deleting them.
>
>
>
> On Thu, Jul 1, 2021 at 9:49 AM Holger Jakobs <holger(at)jakobs(dot)com> wrote:
>
>> Then have a look at the line 377617 and check whether its content is
>> important. The following command shows the line on the terminal. Loading it
>> into an interactive editor seems not to be an option due to the size of the
>> file
>>
>> sed -n '377617 p; 377617 q' /outputfile.csv
>>
>> Then you can decide what to do.
>>
>> Am 01.07.21 um 09:45 schrieb Yambu:
>>
>> Hi Holger
>>
>> There are 8mil lines, the file is not in / , i just removed the other
>> parts.
>>
>> On Thu, Jul 1, 2021 at 9:40 AM Holger Jakobs <holger(at)jakobs(dot)com> wrote:
>>
>>> How many lines has the file? Maybe line 377617 is the last one.
>>>
>>> If so, you can just remove the last line containing this 0x00. The
>>> following command can do this for you:
>>>
>>> sed -i '$ d' /outputfile.csv
>>>
>>> May I remark that having a file in the / directory is a bit awkward?
>>> This directory should not be writable by any user (except root, and root
>>> doesn't fiddle with files for a database).
>>>
>>>
>>>
>>> Am 01.07.21 um 09:29 schrieb Yambu:
>>>
>>> Hello
>>>
>>> How do i get past the error below. I have created a foreign table which
>>> points to a csv file and when i try to create another table it gives error
>>> below
>>>
>>> The csv file has funny characters, how do i bypass those characters.
>>>
>>> create foreign table table1
>>> (
>>> id int,
>>> name text
>>> )
>>> SERVER local_file
>>> OPTIONS (filename '/outputfile.csv' ,
>>> format 'csv',
>>> header 'TRUE',
>>> delimiter ';'
>>> );
>>>
>>>
>>> create table2 as select * from table1;
>>>
>>> ERROR: invalid byte sequence for encoding "UTF8": 0x00
>>> CONTEXT: COPY table1, line 377617
>>>
>>> --
>>> Holger Jakobs, Bergisch Gladbach, Tel. +49-178-9759012
>>>
>>> --
>> Holger Jakobs, Bergisch Gladbach, Tel. +49-178-9759012
>>
>> --
> Holger Jakobs, Bergisch Gladbach, Tel. +49-178-9759012
>
>