Re: Invalid byte sequence

There are a number of such lines, the problem is in identifying all of them
and deleting them.

On Thu, Jul 1, 2021 at 9:49 AM Holger Jakobs <holger(at)jakobs(dot)com> wrote:

> Then have a look at the line 377617 and check whether its content is
> important. The following command shows the line on the terminal. Loading it
> into an interactive editor seems not to be an option due to the size of the
> file
>
> sed -n '377617 p; 377617 q' /outputfile.csv
>
> Then you can decide what to do.
>
> Am 01.07.21 um 09:45 schrieb Yambu:
>
> Hi Holger
>
> There are 8mil lines, the file is not in / , i just removed the other
> parts.
>
> On Thu, Jul 1, 2021 at 9:40 AM Holger Jakobs <holger(at)jakobs(dot)com> wrote:
>
>> How many lines has the file? Maybe line 377617 is the last one.
>>
>> If so, you can just remove the last line containing this 0x00. The
>> following command can do this for you:
>>
>> sed -i '$ d' /outputfile.csv
>>
>> May I remark that having a file in the / directory is a bit awkward? This
>> directory should not be writable by any user (except root, and root doesn't
>> fiddle with files for a database).
>>
>>
>>
>> Am 01.07.21 um 09:29 schrieb Yambu:
>>
>> Hello
>>
>> How do i get past the error below. I have created a foreign table which
>> points to a csv file and when i try to create another table it gives error
>> below
>>
>> The csv file has funny characters, how do i bypass those characters.
>>
>> create foreign table table1
>> (
>> id int,
>> name text
>> )
>> SERVER local_file
>> OPTIONS (filename '/outputfile.csv' ,
>> format 'csv',
>> header 'TRUE',
>> delimiter ';'
>> );
>>
>>
>> create table2 as select * from table1;
>>
>> ERROR: invalid byte sequence for encoding "UTF8": 0x00
>> CONTEXT: COPY table1, line 377617
>>
>> --
>> Holger Jakobs, Bergisch Gladbach, Tel. +49-178-9759012
>>
>> --
> Holger Jakobs, Bergisch Gladbach, Tel. +49-178-9759012
>
>