Re: Using multi-row technique with COPY

On Sun, Nov 27, 2005 at 05:45:31PM -0500, Tom Lane wrote:
> Simon Riggs <simon(at)2ndquadrant(dot)com> writes:
> > COPY FROM can read in sufficient rows until it has a whole block worth
> > of data, then get a new block and write it all with one pair of
> > BufferLock calls.
>
> > Comments?
>
> I don't see any way to do this without horrible modularity violations.
> The COPY code has no business going anywhere near individual buffers;
> for that matter, it doesn't even really know what "a block worth" of
> data is, since the tuples it's dealing with aren't toasted yet.

I wonder if you could get the same effect by having COPY copy into a
temporary table (thus no WAL, no bufferlocks) and then every hundred
rows do a SELECT INTO. Mind you, that just moves the modularity
violation elsewhere since only the bufmgr knows about the size of
buffers really.

Whatever happened to that idea to build as entire datafile with COPY or
some external tool and simply copy it into place and update the
catalog?

Have a nice day,
--
Martijn van Oosterhout <kleptog(at)svana(dot)org> http://svana.org/kleptog/
> Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a
> tool for doing 5% of the work and then sitting around waiting for someone
> else to do the other 95% so you can sue them.