Re: Invalid field size

On 07/04/2017 08:19 AM, Moreno Andreo wrote:
> Il 04/07/2017 16:51, Tom Lane ha scritto:
>> Moreno Andreo <moreno(dot)andreo(at)evolu-s(dot)it> writes:
>>> I've implemented a backup procedure in C# with Npgsql (using COPY TO I
>>> dump all tables in a compressed file) that's been working well in the
>>> last 5 years (and it's still working, since this is a single, isolated
>>> case).
>>> OS: Windows 7
>>> PG: 9.1.6 (I know, it's EOL, but I think it's not matter here)
>>> [ got corrupted data with: ]
>>> 2017-07-04 12:55:27 CEST STATEMENT: COPY
>>> tab(cod,guid,data,blob,thumbnail,descr,type,url,user,home,codrec,table,op,dagg,last)
>>>
>>> FROM STDIN WITH BINARY
>> Pushing binary data around on Windows is always a hazardous proposition.
>> I'd bet something somewhere did a newline format conversion on your
>> data, either adding or removing CR characters. There might not have
>> been any CR or LF bytes in the data fields proper, but it'd be quite
>> plausible for some of the length words used in binary-COPY format to
>> contain such bytes.
>>
>> You might be better off using plain text COPY format; it can withstand
>> this sort of thing much better.
>>
>> regards, tom lane
>>
> When we wrote this function, we first used plain COPY format, but we
> were not satisfied by the file size we got (too big compared to data
> size), so we switched to BINARY (I don't remember if there was also some
> performance matter involved).
> So what you are saying is "in the last 5 years you've been extremely
> lucky?" :-)

Your original post went back and forth on whether you where lucky in the
past:

"... that's been working well in the last 5 years (and it's still
working, since this is a single, isolated case)"

"As for many error I got in the past I assume we are trying to COPY FROM
corrupted data (when using cheap pendrives we get often this error)."

>
> Thanks
> Moreno.
>
>
>

--
Adrian Klaver
adrian(dot)klaver(at)aklaver(dot)com