COPY FROM command v8.1.4

Hi,

Thanks for this quick response.
##Try dumping the second table with COPY TO and diff'ing the dump files to
get more detail about what's missing.##

We tried 3 times from a local file system, instead of the NFS mount (to try
to rule out hardware)

1st attempt: the data copied over fine
2nd attempt: there were 50 records missing
3rd attempt: there were 100 records missing.

Looking back and comparing the data, we realized that its always missing
rows in 25 (24+1) rows sequences or we're calling it a block. So, there were
2 blocks of 25 missing in the 2nd attempt and 4 blocks of 25 missing in the
third. That's the only thing consistent in doing it from anywhere (NFS
mounts or the local filessystem). We've had the same 'sporadic' results on 3
different servers, so that may rule out memory. Our local Reiser FS is in
formated in 64k blocks and our NFS mounts are 4k blocks.

In summary, we are very concerned and have no idea of what direction to go
with this.

~DjK

If you repeat the entire experiment, are the same records missing each time?
I'm wondering about flaky hardware as much as anything.

>From: "Mr. Dan" <bitsandbytes88(at)hotmail(dot)com>
>To: pgsql-admin(at)postgresql(dot)org
>CC: bitsandbytes88(at)hotmail(dot)com
>Subject: COPY FROM command v8.1.4
>Date: Mon, 11 Sep 2006 14:55:26 -0400
>
>We are having some serious problems with the PostgreSQL COPY FROM command
>and we have no clue what to do with this at this point. I think we need to
>ask for 'guru' help in diagnosing this problem. Here is exactly what we are
>doing:
>
>We have a table with 12028587 records in it.
>We do a COPY TO file of the records for this table.
>We count the number of records in the file, it has 12028587 lines.
>We do a COPY FROM that file into a table with identical structure as the
>first table.
>We count the number of records in the second table and there are only
>12028538 records.
>We do another COPY FROM same file onto a different database on an identical
>table and this time there were 12028487 records in the count.
>
>Another symptom of this problem is that when we do a pg_dump to file and
>then a pg_restore from that file (only on large databases 20+ Gb) the
>restore also loses about 100-300 records in the largest tables (the table
>in question has 12+ and 6+ million records). The commands used were:
>
>pg_dump -p <port> -U postgres -Fc -f <file> <db_name>
>pg_restore -p <post> -U postgres -d <db> <file>
>
>We are using PostgreSQL version 8.1.4 with SLES (suse enterprise linux with
>SP3 - 9.3) running on 2 Opteron chips with 16 GB of memory.
>
>Any help in diagnosing or resolving this problem is GREATLY appreciated.
>
>