Re: Syntax error in a large COPY

Hi,

Le Tuesday 06 November 2007 22:40:50 Tom Lane, vous avez écrit :
> Reg Me Please <regmeplease(at)gmail(dot)com> writes:
> > I'm trying to understand what and where.
> > The point is that I have this 29M+ lines script telling me there's a
> > problem somewhere.

You could use pgloader, which reads CSV input (or text which is not quite CSV)
and will import valid rows and reject invalid ones, filling both a reject log
file with error messages and a rejected data file with bad input lines.
Basically, it will issue the dichotomy steps (branch and bound) for you.
http://pgfoundry.org/projects/pgloader
http://pgloader.projects.postgresql.org/

> It told you exactly where it detected the problem.
>
> Actually ... are you looking at the right error message? If this is a
> copy operation within a larger psql script, I think it's probably true
> that psql will point at the end of the copy data, because *it* doesn't
> know any better. But the error message from the backend should
> correctly finger which line of copy input it got confused at.

Or just use the server logged error message, which will be the same as
reported by pgloader (no magic here) --- pgloader will import successfully
the rest of the data and have the faulty lines separated out in the reject
file, but still uses COPY internally.

Hope this helps,
--
dim