Re: COPY enhancements

Josh Berkus wrote:
>> The performance of every path to get data into the database besides COPY
>> is too miserable for us to use anything else, and the current
>> inflexibility makes it useless for anything but the cleanest input data.
>
> One potential issue we're facing down this road is that current COPY has
> a dual purpose: for database restore, and for importing and exporting
> data. At some point, we may want to separate those two behaviors,
> because we'll be adding bells and fringes to import/export which slow
> down overall performance or add bugs.

+1. There is an infinite number of bells and whistles we could add to
COPY, and there's also a number of further optimizations that would make
the loading faster. But the code is quite a mess already, because it's
already highly optimized at the expense of readibility. We need to
separate the input parsing from the fast bulk insertion.

Letting external modules replace the input parsing part would allow you
to a write parser for any input format you like. You could even get the
input from a different source altogether, like from another database via
dblink, in a binary format of some sort.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com