Re: exposing COPY API

On 02/07/2011 11:34 AM, Andrew Dunstan wrote:
>
>
> On 02/04/2011 05:49 AM, Itagaki Takahiro wrote:
>> Here is a demonstration to support jagged input files. It's a patch
>> on the latest patch. The new added API is:
>>
>> bool NextLineCopyFrom(
>> [IN] CopyState cstate,
>> [OUT] char ***fields, [OUT] int *nfields, [OUT] Oid *tupleOid)
>>
>> It just returns separated fields in the next line. Fortunately, I need
>> no extra code for it because it is just extracted from NextCopyFrom().
>>
>> I'm willing to include the change into copy APIs,
>> but we still have a few issues. See below.
>
>
> I've looked at this, and I think it will do what I want. I haven't had
> time to play with it, although I hope to soon. AIUI, it basically
> hands back the raw parsed strings to the user, who then has the
> responsibility of constructing Datum and Nulls arrays to form the
> tuple. That should be all I need. So +1 from me for including it. In
> fact, +10. And many thanks.
>
>
> I think we need a better name though. NextCopyFromRawFields maybe.

Here is a patch against the latest revision of file_fdw to exercise this
API. It includes some regression tests, and I think apart from one or
two small details plus a requirement for documentation, is complete.

This work is also published at
<https://github.com/adunstan/postgresql-dev/tree/sqlmed3>

Here's an excerpt from the regression tests:

CREATE FOREIGN TABLE jagged_text (
t text[]
) SERVER file_server
OPTIONS (format 'csv', filename
'/home/andrew/pgl/pg_head/contrib/file_fdw/data/jagged.csv', header
'true', textarray 'true');
SELECT * FROM jagged_text;
t
--------------------------------------------
{"one field"}
{two,fields}
{three,NULL,"fields of which one is null"}
{"next line has no fields"}
{}
(5 rows)

SELECT t[3] AS a, t[1] AS b, t[99] AS c FROM jagged_text;
a | b | c
-----------------------------+-------------------------+---
| one field |
| two |
fields of which one is null | three |
| next line has no fields |
| |
(5 rows)

Overall, this API works quite nicely, and does exactly what I want, so
again many thanks.

cheers

andrew