From: | danap <danap(at)itstriangle(dot)com> |
---|---|
To: | albertwang <wang_yanlei(at)yahoo(dot)com>, PostgreSQL JDBC <pgsql-jdbc(at)postgresql(dot)org> |
Subject: | Re: Streaming ResultSet |
Date: | 2017-06-23 16:34:39 |
Message-ID: | 594D431F.9000305@itstriangle.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-jdbc |
albertwang wrote:
> Hi Kevin,
>
> How is your question / suggestion going?
>
> I met the same problem. Our company uses PostgreSQL as the OLTP system. I am
> working on an ETL project which reads data from a PostgreSQL db, transforms
> the data and then load the data into another PostgreSQL db. The ETL tool
> uses JDBC.
>
> My problem is that the ETL won't start to transform the data before the
> whole data result set has been loaded into memory.
>
> I would like to do the data transforming and loading while extracting data.
How about instead of using the JDBC intermediary, extract the data into a local
file/memory db, then transform and load from it to the destination PostgreSQL
db. Set up three seperate parallel processes to perform the task.
danap.
>
> Do you know if there is any progress reagrding the streaming result set? As
> I know, MySQL does support it.
>
> Thank you.
>
> Regards,
> Albert
From | Date | Subject | |
---|---|---|---|
Next Message | Dave Cramer | 2017-06-26 13:34:05 | [pgjdbc/pgjdbc] cd400f: add missing connection documentation, fix spelling... |
Previous Message | Thomas Kellerer | 2017-06-23 05:42:15 | Re: Streaming ResultSet |