Re: Remote troubleshooting session connection?

On Sat, Apr 5, 2014 at 6:41 AM, Jim Garrison <jim(dot)garrison(at)nwea(dot)org> wrote:

> An ETL "job" runs inside its own transaction and consists of a series of
> queries that transform the data from staging tables to the destination
> tables. If a failure occurs, the transaction rolls back so there's no
> "debris" left over -- which makes troubleshooting very difficult.
>

If you are loading huge amount of data then:

1) Committing every 10000 (or so) rows might make sense
2) Have you considered using COPY API in Postgres' JDBC?
3) Which version of PostgreSQL are you using? I guess 9.3 has a freeze
option which might help you. I am not sure if the API supports it.

Best Regards,

*Sameer Kumar | Database Consultant*

*ASHNIK PTE. LTD.*

101 Cecil Street, #11-11 Tong Eng Building, Singapore 069533

M: *+65 8110 0350* T: +65 6438 3504 | www.ashnik.com

*[image: icons]*

[image: Email patch] <http://www.ashnik.com/>

This email may contain confidential, privileged or copyright material and
is solely for the use of the intended recipient(s).