From: | Bruce Momjian <bruce(at)momjian(dot)us> |
---|---|
To: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
Cc: | PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: COPY and file_fdw with fixed column widths |
Date: | 2015-04-28 19:50:00 |
Message-ID: | 20150428195000.GA31727@momjian.us |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On Tue, Apr 28, 2015 at 12:46:22PM -0700, Tom Lane wrote:
> Bruce Momjian <bruce(at)momjian(dot)us> writes:
> > I know COPY doesn't support importing files with fixed column widths,
> > i.e. we can't say field1 is the first 30 characters, and field2 is the
> > rest of the line. We need a unique delimiter at column 31. (Commercial
> > Ingres does support this ability.)
>
> > I know we tell most people to use sed, Perl, or an ETL tool to convert
> > files into a format COPY understands, and I think that is a reasonable
> > answer. However, the file_fdw also reads our COPY format, and in that
> > case, the data file might be updated regularly and running an ETL
> > process on it every time it is read is inconvenient.
>
> COPY is, and has always been intended to be, as fast as possible; loading
> format transformation abilities onto it seems like a fundamental mistake.
> Therefore, if you wish file_fdw were more flexible, I think the answer is
> to create a variant of file_fdw that doesn't use COPY but some other
> mechanism.
Yes, I think this is a missing feature. While we can tell people to do
ETL for loading, we are really not doing that for file_fdw.
--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ Everyone has their own god. +
From | Date | Subject | |
---|---|---|---|
Next Message | Tom Lane | 2015-04-28 19:50:04 | Re: FIX : teach expression walker about RestrictInfo |
Previous Message | Tom Lane | 2015-04-28 19:46:22 | Re: COPY and file_fdw with fixed column widths |