bizgres - bizgres: Enhancements and Fixes 1.

From: nwakefield(at)pgfoundry(dot)org (User Nwakefield)
To: pgsql-committers(at)postgresql(dot)org
Subject: bizgres - bizgres: Enhancements and Fixes 1.
Date: 2006-04-24 06:14:25
Message-ID: 20060424061425.C5E1986C5FB@pgfoundry.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-committers

Log Message:
-----------
Enhancements and Fixes
1. NIOFileReader has been reworked behind the scenes to solve a severe problem with record delimiters existing within a field. From now on the character stream is read field by field, with an unsynchronized buffer existing between the file channel and the reading class. Performance should be comparable, with complete support for multi character delimiters.
2. FastSimpleDateFormat, had a problem with its internal cache that resulted in fields with the same value matching even though they were not in the same position
3. Parameter write back from the SQL JOb executor has been migrated from MPP. This allow for SQL jobs to writeback to parameter list. See the IncrementalFetch.xml for an example of its usage.
4. Added RunNIOFileReaderTestjobs.java, this is a simple JUnit test suite that tests the reader.

Modified Files:
--------------
bizgres/KETL/src/com/kni/etl/dbutils:
Upsert.java (r1.3 -> r1.4)
(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/dbutils/Upsert.java.diff?r1=1.3&r2=1.4)
DatabaseColumnDefinition.java (r1.2 -> r1.3)
(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/dbutils/DatabaseColumnDefinition.java.diff?r1=1.2&r2=1.3)
ColumnDefinition.java (r1.2 -> r1.3)
(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/dbutils/ColumnDefinition.java.diff?r1=1.2&r2=1.3)
bizgres/KETL/src/com/kni/etl:
BaseDataItem.java (r1.2 -> r1.3)
(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/BaseDataItem.java.diff?r1=1.2&r2=1.3)
DataItem.java (r1.2 -> r1.3)
(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/DataItem.java.diff?r1=1.2&r2=1.3)
SourceFieldDefinition.java (r1.1 -> r1.2)
(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/SourceFieldDefinition.java.diff?r1=1.1&r2=1.2)
EngineConstants.java (r1.6 -> r1.7)
(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/EngineConstants.java.diff?r1=1.6&r2=1.7)
SQLJob.java (r1.4 -> r1.5)
(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/SQLJob.java.diff?r1=1.4&r2=1.5)
SQLJobExecutor.java (r1.3 -> r1.4)
(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/SQLJobExecutor.java.diff?r1=1.3&r2=1.4)
FastInputChannel.java (r1.2 -> r1.3)
(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/FastInputChannel.java.diff?r1=1.2&r2=1.3)
bizgres/KETL/src/com/kni/etl/ketl/writer:
SubComponentParallelBatchWriter.java (r1.1 -> r1.2)
(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/ketl/writer/SubComponentParallelBatchWriter.java.diff?r1=1.1&r2=1.2)
JDBCWriter.java (r1.6 -> r1.7)
(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/ketl/writer/JDBCWriter.java.diff?r1=1.6&r2=1.7)
PGBulkWriter.java (r1.4 -> r1.5)
(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/ketl/writer/PGBulkWriter.java.diff?r1=1.4&r2=1.5)
ETLWriter.java (r1.2 -> r1.3)
(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/ketl/writer/ETLWriter.java.diff?r1=1.2&r2=1.3)
SubComponentBatchRunnerThread.java (r1.1 -> r1.2)
(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/ketl/writer/SubComponentBatchRunnerThread.java.diff?r1=1.1&r2=1.2)
bizgres/KETL/src/com/kni/etl/stringtools:
FastSimpleDateFormat.java (r1.1 -> r1.2)
(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/stringtools/FastSimpleDateFormat.java.diff?r1=1.1&r2=1.2)
bizgres/KETL/src/com/kni/etl/ketl:
ETLStep.java (r1.6 -> r1.7)
(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/ketl/ETLStep.java.diff?r1=1.6&r2=1.7)
bizgres/KETL/src/com/kni/etl/ketl/reader:
NIOFileReader.java (r1.5 -> r1.6)
(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/ketl/reader/NIOFileReader.java.diff?r1=1.5&r2=1.6)

Added Files:
-----------
bizgres/KETL/src/com/kni/etl:
FieldLevelFastInputChannel.java (r1.1)
(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/FieldLevelFastInputChannel.java?rev=1.1&content-type=text/x-cvsweb-markup)
bizgres/KETL/src/com/kni/etl/ketl/test:
RunNIOFileReaderTestJobs.java (r1.1)
(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/ketl/test/RunNIOFileReaderTestJobs.java?rev=1.1&content-type=text/x-cvsweb-markup)

Browse pgsql-committers by date

  From Date Subject
Next Message User Nwakefield 2006-04-24 06:19:04 bizgres - bizgres: New Directory
Previous Message User Nwakefield 2006-04-24 06:11:18 bizgres - bizgres: New Directory