Re: making postgresql "source" its data feed from external csv file ...

From: Albretch Mueller <lbrtchx(at)gmail(dot)com>
To: pgsql-jdbc(at)postgresql(dot)org
Cc: Craig Ringer <craig(at)postnewspapers(dot)com(dot)au>
Subject: Re: making postgresql "source" its data feed from external csv file ...
Date: 2011-01-04 01:09:14
Message-ID: AANLkTim3AAMP=nHZMp877audLYcZtA-vEzcNURgTJHdA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-jdbc

On Tue, Jan 4, 2011 at 12:14 AM, Craig Ringer
<craig(at)postnewspapers(dot)com(dot)au> wrote:
> On 01/04/2011 09:55 AM, Albretch Mueller wrote:
>
>>  As I understand postgresql "COPY FROM" does in effect copy the data
>> to the DBMS, but what I need is for the DBMS to "source" its data feed
>> with the outer file and synchronize it as it changes without having to
>> run "COPY TO". Is this possible? This way I can keep the data in a way
>> that I can try on any DBMS
>
> No, that's not currently possible. What you describe is covered by SQL/MED.
> There's ongoing work to implement SQL/MED for 9.1 / 9.2, but there is no
> support for it in any current release version of PostgreSQL. SQL/MED is
> intended for a lot more than just accessing flat files, but should be
> capable of that.
>
> There's a reason relational databases don't store their backing data as CSV
> files - because they're *SLOW* to work with, in particular to update. You
> can expect horribly slow write performance with any scheme like this even if
> you do get it working.
~
Well, yes, writing to an inmem db while coding should be fast enough
I guess ;-)
~
I thought of something else a few moments ago. Is there a way to
register to postgresql events like you do with java.awt., javax.swing.
and j2ee management events and listen to them?
~
If you could somehow do that with jdbc, you could register to "COPY
FROM" events to certain files and regularly synchronize the data.
Certain schedule based on a measurable degree of dirtiness, time
stamps (back up incremental deltas every Tuesdays night ...) and data
semantics (if the increment or the total amount of monies reach some
value …) would help
~
I think I will have to do with "COPY [FROM|TO]" for now until SQL/MED
~
http://wiki.postgresql.org/wiki/SQL/MED
~
http://wiki.postgresql.org/images/4/4c/SQLMED-FOSDEM2009.pdf
~
becomes a viable technology
~
Thanks
lbrtchx

In response to

Browse pgsql-jdbc by date

  From Date Subject
Next Message Andreas Joseph Krogh 2011-01-04 11:41:11 Add URL-property to PGXADataSource
Previous Message Maciek Sakrejda 2011-01-04 00:38:29 Re: making postgresql "source" its data feed from external csv file ...