Re: COPY enhancements

Hi all,

I think there is a misunderstanding about what the current patch is about.
The patch includes 2 things:
- error logging in a table for bad tuples in a COPY operation (see
http://wiki.postgresql.org/wiki/Error_logging_in_COPY for an example;
the error message, command and so on are automatically logged)
- auto-partitioning in a hierarchy of child table if the COPY targets a
parent table.

The patch does NOT include:
- logging errors into a file (a feature we can add later on (next commit
fest?))

I can put the auto-partitioning in a separate patch if that helps but
this patch relies on error logging to log possible errors in the routing
of tuples.

I think that the way to move forward is first to have a basic error
logging facility that logs into a database table (like the current patch
does) and then add enhancements. I don't think we should try to do the
file logging at the same time. If you prefer to postpone the
auto-partitioning to the next commit fest, I can strip it from the
current patch and re-submit it for the next fest (but it's just 2
isolated methods really easy to review).

Emmanuel

Robert Haas wrote:
> On Wed, Oct 7, 2009 at 3:17 AM, Greg Smith <gsmith(at)gregsmith(dot)com> wrote:
>
>> I know this patch is attracting more reviewers lately, is anyone tracking
>> the general architecture of the code yet? Emmanuel's work is tough to
>> review just because there's so many things mixed together, and there's other
>> inputs I think should be considered at the same time while we're all testing
>> in there (such as the COPY patch Andrew Dunstan put together).
>>
>
> I hadn't realized this was an issue, but I think it's a good point: a
> patch that does one thing well is much more likely to get accepted
> than a patch that does two things well, let alone two things poorly.
> It's just much easier to review and verify. Or maybe the name of the
> patch maybe should have tipped me off: "COPY enhancements" vs. "make
> COPY have feature X".
>
>
>> What I'd like to see is for everything to get broken more into component
>> chunks that can get commited and provide something useful one at a time,
>> because I doubt taskmaster Robert is going to let this one linger around
>> with scope creep for too long before being pushed out to the next
>> CommitFest.
>>
>
> I'm can't decide whether to feel good or bad about that appelation, so
> I'm going with both. But in all seriousness if this patch needs
> substantial reworking (which it sounds like it does) we should
> postpone it to the next CF; we are quickly running out of days, and
> it's not fair to reviewers or committers to ask for new reviews of
> substantially revised code with a only a week to go.
>
> ...Robert
>

--
Emmanuel Cecchet
Aster Data Systems
Web: http://www.asterdata.com