Re: Cannot import logs from csv

Something else that causes a similar error is if the PostgreSQL daemon is restarted in the middle of logging a query. The log line is truncated and the next log line starts straight after the truncated line without even a newline in between. The problematic line can be many lines before where the import process shows an error.

The last problem I found that shows this error is when control characters end up in your queries, usually because someone isn't validating their input >:-( Putting the logfiles through "iconv -c -f UTF8 -t ASCII" seems to fix this.

--
Helen Griffiths

> -----Original Message-----
> From: pgsql-general-owner(at)postgresql(dot)org [mailto:pgsql-general-
> owner(at)postgresql(dot)org] On Behalf Of Helen Griffiths
> Sent: 18 October 2013 18:27
> To: pgsql-general(at)postgresql(dot)org
> Subject: Re: [GENERAL] Cannot import logs from csv
>
> On Fri, 18 Oct 2013, Adrian Klaver wrote:
>
> > This came up before recently in this thread:
> >
> > http://www.postgresql.org/message-
> id/CADK3HHJNEWKD9gNyXmjv9ABbn+37rY3M
> > vp9=1j7msg9YpoBBBw(at)mail(dot)gmail(dot)com
> >
> > To cut to the chase, in that case the OP found:
> >
> > "Ok, I found the offending line. It was not the pgadmin line. There
> > was a line with a large binary insert."
> >
> > Not sure if that helps.
>
> It might: there are then some lines like:
> 2013-10-11 02:44:26.164
> BST,"postgres","course_records",19216,"[local]",525740b3.4b10,124887,"idle
> in transaction",2013-10-11 01:05:07 BST,7/16388,0,LOG,00000,"fastpath
> function call: ""loread"" (OID 954)",,,,,,,,
>
> I suspect the ""loread"" is the offender.
>
> Thanks to all for the help. I would never have thought to suspect the lines after
> the one given in the error message.
>
> --
> Helen Griffiths
>
>
> --
> Sent via pgsql-general mailing list (pgsql-general(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-general