[BACKLOG-6461] Oracle repository: Impossible to create CSV datasource… by ivanpogodin · Pull Request #727 · pentaho/data-access

ivanpogodin · 2016-04-14T09:10:52Z

… where column names in csv contain - and / which get converted to text, HYPHEN and DIVIDED_BY, causing column name in table to be too long [ORA-00972 - Identifier too long].

ivanpogodin · 2016-04-14T09:12:49Z

@mdamour1976 , @pentaho-nbaker , could you review it?

brosander · 2016-04-14T10:55:44Z

Build Commands:

Build:

ant -Dtestreports.dir=bin/reports/unit-test -Djunit.jvmargs=-XX:PermSize=256m -f build.xml clean-all resolve jacoco && ant -Dpentaho.coding.standards.version=1.0.4 -Dtestreports.dir=bin/reports/integration-test -Djunit.jvmargs=-XX:PermSize=256m -f build.xml jacoco-integration checkstyle

Cleanup:

rm -r ~/.ivy2/local || echo no publish local to remove

Changed files:

src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/CsvTransformGenerator.java
src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/StagingTransformGenerator.java
test-src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/CsvTransformGeneratorTest.java

Unit test coverage change:

org.pentaho.platform.dataaccess.datasource.wizard.models.ModelInfo:

Complexity Change : + 1.64%
Instruction Change : + .70%
Line Change : + .83%
Method Change : + 4.00%

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.CsvTransformGenerator:

Branch Change : + 30.30%
Complexity Change : + 7.78%
Instruction Change : + 21.32%
Line Change : + 22.26%

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.StagingTransformGenerator:

Complexity Change : -.18%
Instruction Change : -.03%
Line Change : -.05%
Method Change : -.91%

Integration test coverage change:

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.CsvTransformGenerator:

Branch Change : -50.51%
Complexity Change : -41.11%
Instruction Change : -42.84%
Line Change : -37.95%
Method Change : -13.89%

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.StagingTransformGenerator:

Complexity Change : + .53%
Instruction Change : + .09%
Line Change : + .11%
Method Change : + .60%

… where column names in csv contain - and / which get converted to text, HYPHEN and DIVIDED_BY, causing column name in table to be too long [ORA-00972 - Identifier too long].

ivanpogodin · 2016-04-15T14:40:02Z

@brosander , @mdamour1976 , @pentaho-nbaker , could you review it?
The code coverage is improved

ivanpogodin · 2016-04-19T12:14:37Z

@mdamour1976 , @pentaho-nbaker , could you review it?

ivanpogodin · 2016-04-20T12:05:19Z

@brosander , @mdamour1976 , @pentaho-nbaker , could you review it?

ivanpogodin · 2016-04-21T12:02:46Z

@brosander , @mdamour1976 , @pentaho-nbaker , could you review it?

ivanpogodin · 2016-04-22T12:36:18Z

@brosander , @mdamour1976 , @pentaho-nbaker , could you review it?

brosander · 2016-04-26T09:15:31Z

Build Commands:

Build:

ant -Dtestreports.dir=bin/reports/unit-test -Djunit.jvmargs=-XX:PermSize=256m -f build.xml clean-all resolve jacoco && ant -Dpentaho.coding.standards.version=1.0.4 -Dtestreports.dir=bin/reports/integration-test -Djunit.jvmargs=-XX:PermSize=256m -f build.xml jacoco-integration checkstyle

Cleanup:

rm -r ~/.ivy2/local || echo no publish local to remove

Changed files:

src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/CsvTransformGenerator.java
src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/StagingTransformGenerator.java
test-src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/CsvTransformGeneratorIT.java
test-src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/CsvTransformGeneratorTest.java

Unit test coverage change:

org.pentaho.platform.dataaccess.datasource.wizard.models.ModelInfo:

Complexity Change : + 1.64%
Instruction Change : + .70%
Line Change : + .83%
Method Change : + 4.00%

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.CsvTransformGenerator:

Branch Change : + 31.25%
Complexity Change : + 10.61%
Instruction Change : + 21.11%
Line Change : + 21.91%

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.StagingTransformGenerator:

Complexity Change : -.18%
Instruction Change : -.03%
Line Change : -.05%
Method Change : -.91%

Integration test coverage change:

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.CsvTransformGenerator:

Branch Change : -2.78%
Complexity Change : -8.59%
Instruction Change : -3.39%
Line Change : -.17%
Method Change : + 2.78%

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.StagingTransformGenerator:

Complexity Change : + .53%
Instruction Change : + .09%
Line Change : + .11%
Method Change : + .60%

pamval · 2016-04-26T09:17:36Z

CsvTransformGenerator Integration test coverage has dropped significantly. Can you review please, @ivanpogodin ?

ivanpogodin · 2016-04-28T13:49:34Z

@pamval , could you review it, please?

wingman-pentaho · 2016-04-28T14:07:18Z

Build Commands:

Build:

ant -Dtestreports.dir=bin/reports/unit-test -Djunit.jvmargs=-XX:PermSize=256m -f build.xml clean-all resolve jacoco && ant -Dpentaho.coding.standards.version=1.0.4 -Dtestreports.dir=bin/reports/integration-test -Djunit.jvmargs=-XX:PermSize=256m -f build.xml jacoco-integration checkstyle

Cleanup:

rm -r ~/.ivy2/local || echo no publish local to remove

Changed files:

src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/CsvTransformGenerator.java
src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/StagingTransformGenerator.java
test-src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/CsvTransformGeneratorIT.java
test-src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/CsvTransformGeneratorTest.java
test-src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/StagingTransformGeneratorTest.java

Unit test coverage change:

org.pentaho.platform.dataaccess.datasource.wizard.models.ModelInfo:

Complexity Change : + 1.64%
Instruction Change : + .70%
Line Change : + .83%
Method Change : + 4.00%

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.CsvTransformGenerator:

Branch Change : + 32.81%
Complexity Change : + 12.88%
Instruction Change : + 21.11%
Line Change : + 21.91%

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.StagingTransformGenerator:

Branch Change : + 4.00%
Complexity Change : + 3.48%
Instruction Change : + 1.38%
Line Change : + 1.74%
Method Change : + 2.22%

Integration test coverage change:

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.CsvTransformGenerator:

Branch Change : + .35%
Complexity Change : -4.04%
Instruction Change : -1.61%
Line Change : + .91%
Method Change : + 2.78%

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.StagingTransformGenerator:

Complexity Change : + .53%
Instruction Change : + .09%
Line Change : + .11%
Method Change : + .60%

org.pentaho.platform.dataaccess.datasource.wizard.sources.csv.FileTransformStats:

Branch Change : + 50.00%
Complexity Change : + 3.45%
Instruction Change : + 9.03%
Line Change : + 5.36%
Method Change : + 3.70%

pamval · 2016-04-28T14:35:54Z

@mdamour1976 @e-cuellar, can you take a look at this one ? thanks!

e-cuellar · 2016-04-28T16:36:24Z

@pmalves The pull request failed some checks at Wingman...

ivanpogodin · 2016-04-29T11:34:41Z

@pamval , @mdamour1976 , @e-cuellar , could you review this, please?

ivanpogodin · 2016-05-04T06:07:59Z

@pamval , @mdamour1976 , @e-cuellar , could you review this, please?

ivanpogodin · 2016-05-05T12:18:04Z

@pamval , @mdamour1976 , @e-cuellar , could you review this, please?

wingman-pentaho · 2016-05-05T12:54:36Z

Build Commands:

Build:

ant -Dtestreports.dir=bin/reports/unit-test -Djunit.jvmargs=-XX:PermSize=256m -f build.xml clean-all resolve jacoco && ant -Dpentaho.coding.standards.version=1.0.4 -Dtestreports.dir=bin/reports/integration-test -Djunit.jvmargs=-XX:PermSize=256m -f build.xml jacoco-integration checkstyle

Cleanup:

rm -r ~/.ivy2/local || echo no publish local to remove

Changed files:

src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/CsvTransformGenerator.java
src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/StagingTransformGenerator.java
test-src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/CsvTransformGeneratorIT.java
test-src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/CsvTransformGeneratorTest.java
test-src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/StagingTransformGeneratorTest.java

Newly Broken Tests:

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.CsvTransformGeneratorTest.testGetMaxColumnNameLength:

java.lang.NullPointerException

java.lang.NullPointerException
    at org.pentaho.di.core.database.Database.getObjectName(Database.java:4507)
    at org.pentaho.di.core.logging.LoggingObject.grabLoggingObjectInformation(LoggingObject.java:144)
    at org.pentaho.di.core.logging.LoggingObject.<init>(LoggingObject.java:56)
    at org.pentaho.di.core.logging.LoggingRegistry.registerLoggingSource(LoggingRegistry.java:65)
    at org.pentaho.di.core.logging.LogChannel.<init>(LogChannel.java:58)
    at org.pentaho.di.core.database.Database.<init>(Data

Unit test coverage change:

org.pentaho.platform.dataaccess.datasource.wizard.models.ModelInfo:

Complexity Change : + 1.64%
Instruction Change : + .70%
Line Change : + .83%
Method Change : + 4.00%

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.CsvTransformGenerator:

Branch Change : + 40.62%
Complexity Change : + 17.42%
Instruction Change : + 26.32%
Line Change : + 28.39%
Method Change : + 8.33%

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.StagingTransformGenerator:

Branch Change : + 4.00%
Complexity Change : + 3.48%
Instruction Change : + 1.38%
Line Change : + 1.74%
Method Change : + 2.22%

Integration test coverage change:

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.CsvTransformGenerator:

Branch Change : + 3.47%
Complexity Change : + 2.78%
Instruction Change : + .16%
Line Change : + 4.69%
Method Change : + 11.11%

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.StagingTransformGenerator:

Complexity Change : + 1.75%
Instruction Change : + 1.03%
Line Change : + 1.01%
Method Change : + 3.73%

org.pentaho.platform.dataaccess.datasource.wizard.sources.csv.FileTransformStats:

Branch Change : + 50.00%
Complexity Change : + 3.45%
Instruction Change : + 9.03%
Line Change : + 5.36%
Method Change : + 3.70%

pamval · 2016-05-05T12:56:53Z

@ivanpogodin, broken test in wingman. Can you take a look ?

wingman-pentaho · 2016-05-06T11:12:25Z

Build Commands:

Build:

ant -Dtestreports.dir=bin/reports/unit-test -Djunit.jvmargs=-XX:PermSize=256m -f build.xml clean-all resolve jacoco && ant -Dpentaho.coding.standards.version=1.0.4 -Dtestreports.dir=bin/reports/integration-test -Djunit.jvmargs=-XX:PermSize=256m -f build.xml jacoco-integration checkstyle

Cleanup:

rm -r ~/.ivy2/local || echo no publish local to remove

Changed files:

src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/CsvTransformGenerator.java
src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/StagingTransformGenerator.java
test-src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/CsvTransformGeneratorIT.java
test-src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/CsvTransformGeneratorTest.java
test-src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/StagingTransformGeneratorTest.java

Newly Broken Tests:

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.CsvTransformGeneratorTest.testGetMaxColumnNameLength:

java.lang.NullPointerException

java.lang.NullPointerException
    at org.pentaho.di.core.database.Database.getObjectName(Database.java:4507)
    at org.pentaho.di.core.logging.LoggingObject.grabLoggingObjectInformation(LoggingObject.java:144)
    at org.pentaho.di.core.logging.LoggingObject.<init>(LoggingObject.java:56)
    at org.pentaho.di.core.logging.LoggingRegistry.registerLoggingSource(LoggingRegistry.java:65)
    at org.pentaho.di.core.logging.LogChannel.<init>(LogChannel.java:58)
    at org.pentaho.di.core.database.Database.<init>(Data

Unit test coverage change:

org.pentaho.platform.dataaccess.datasource.wizard.models.ModelInfo:

Complexity Change : + 1.64%
Instruction Change : + .70%
Line Change : + .83%
Method Change : + 4.00%

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.CsvTransformGenerator:

Branch Change : + 40.62%
Complexity Change : + 17.42%
Instruction Change : + 26.32%
Line Change : + 28.39%
Method Change : + 8.33%

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.StagingTransformGenerator:

Branch Change : + 4.00%
Complexity Change : + 3.48%
Instruction Change : + 1.38%
Line Change : + 1.74%
Method Change : + 2.22%

Integration test coverage change:

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.CsvTransformGenerator:

Branch Change : + 3.47%
Complexity Change : + 2.78%
Instruction Change : + .16%
Line Change : + 4.69%
Method Change : + 11.11%

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.StagingTransformGenerator:

Complexity Change : + 1.75%
Instruction Change : + 1.03%
Line Change : + 1.01%
Method Change : + 3.73%

org.pentaho.platform.dataaccess.datasource.wizard.sources.csv.FileTransformStats:

Branch Change : + 50.00%
Complexity Change : + 3.45%
Instruction Change : + 9.03%
Line Change : + 5.36%
Method Change : + 3.70%

wingman-pentaho · 2016-05-06T13:22:50Z

Build Commands:

Build:

ant -Dtestreports.dir=bin/reports/unit-test -Djunit.jvmargs=-XX:PermSize=256m -f build.xml clean-all resolve jacoco && ant -Dpentaho.coding.standards.version=1.0.4 -Dtestreports.dir=bin/reports/integration-test -Djunit.jvmargs=-XX:PermSize=256m -f build.xml jacoco-integration checkstyle

Cleanup:

rm -r ~/.ivy2/local || echo no publish local to remove

Changed files:

src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/CsvTransformGenerator.java
src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/StagingTransformGenerator.java
test-src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/CsvTransformGeneratorIT.java
test-src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/CsvTransformGeneratorTest.java
test-src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/StagingTransformGeneratorTest.java

Unit test coverage change:

org.pentaho.platform.dataaccess.datasource.wizard.models.ModelInfo:

Complexity Change : + 1.64%
Instruction Change : + .70%
Line Change : + .83%
Method Change : + 4.00%

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.CsvTransformGenerator:

Branch Change : + 45.31%
Complexity Change : + 19.70%
Instruction Change : + 31.05%
Line Change : + 31.64%
Method Change : + 8.33%

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.StagingTransformGenerator:

Branch Change : + 4.00%
Complexity Change : + 3.48%
Instruction Change : + 1.38%
Line Change : + 1.74%
Method Change : + 2.22%

Integration test coverage change:

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.CsvTransformGenerator:

Branch Change : + 3.47%
Complexity Change : + 2.78%
Instruction Change : + .16%
Line Change : + 4.69%
Method Change : + 11.11%

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.StagingTransformGenerator:

Complexity Change : + 1.75%
Instruction Change : + 1.03%
Line Change : + 1.01%
Method Change : + 3.73%

org.pentaho.platform.dataaccess.datasource.wizard.sources.csv.FileTransformStats:

Branch Change : + 50.00%
Complexity Change : + 3.45%
Instruction Change : + 9.03%
Line Change : + 5.36%
Method Change : + 3.70%

ivanpogodin · 2016-05-06T13:36:24Z

@pamval , @mdamour1976 , @e-cuellar , could you review this, please?

ivanpogodin · 2016-05-10T11:59:00Z

@pamval , @mdamour1976 , @e-cuellar , could you review this, please?
All checks have passed

pamval · 2016-05-11T10:04:08Z

@e-cuellar can you take a look ?

buildguy · 2023-01-27T18:01:59Z

❌ Build failed in 30m 39s

Build command:

mvn clean verify -B -e -Daudit -amd

❗ No tests found!

Errors:

Filtered log (click to expand)

/build/wingman/workspace/maintenance/wingman/builds/pentaho.data-access.PR-727/pom.xml does not exist.

ℹ️ This is an automatic message

[BACKLOG-6461] Oracle repository: Impossible to create CSV datasource…

afe85db

… where column names in csv contain - and / which get converted to text, HYPHEN and DIVIDED_BY, causing column name in table to be too long [ORA-00972 - Identifier too long].

ivanpogodin force-pushed the BACKLOG-6461 branch from fa5042f to afe85db Compare April 15, 2016 14:38

[BACKLOG-6461] add tests

2580955

ivanpogodin force-pushed the BACKLOG-6461 branch from d2a6baf to 270741d Compare May 6, 2016 10:59

[BACKLOG-6461] more tests

5631d09

ivanpogodin force-pushed the BACKLOG-6461 branch from 270741d to 5631d09 Compare May 6, 2016 13:13

Conversation

ivanpogodin commented Apr 14, 2016

Uh oh!

ivanpogodin commented Apr 14, 2016

Uh oh!

brosander commented Apr 14, 2016

Build Commands:

Build:

Cleanup:

Changed files:

Unit test coverage change:

Integration test coverage change:

Uh oh!

ivanpogodin commented Apr 15, 2016

Uh oh!

ivanpogodin commented Apr 19, 2016

Uh oh!

ivanpogodin commented Apr 20, 2016

Uh oh!

ivanpogodin commented Apr 21, 2016

Uh oh!

ivanpogodin commented Apr 22, 2016

Uh oh!

brosander commented Apr 26, 2016

Build Commands:

Build:

Cleanup:

Changed files:

Unit test coverage change:

Integration test coverage change:

Uh oh!

pamval commented Apr 26, 2016

Uh oh!

ivanpogodin commented Apr 28, 2016

Uh oh!

wingman-pentaho commented Apr 28, 2016

Build Commands:

Build:

Cleanup:

Changed files:

Unit test coverage change:

Integration test coverage change:

Uh oh!

pamval commented Apr 28, 2016

Uh oh!

e-cuellar commented Apr 28, 2016

Uh oh!

ivanpogodin commented Apr 29, 2016

Uh oh!

ivanpogodin commented May 4, 2016

Uh oh!

ivanpogodin commented May 5, 2016

Uh oh!

wingman-pentaho commented May 5, 2016

Build Commands:

Build:

Cleanup:

Changed files:

Newly Broken Tests:

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.CsvTransformGeneratorTest.testGetMaxColumnNameLength:

Unit test coverage change:

Integration test coverage change:

Uh oh!

pamval commented May 5, 2016

Uh oh!

wingman-pentaho commented May 6, 2016

Build Commands:

Build:

Cleanup:

Changed files:

Newly Broken Tests:

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.CsvTransformGeneratorTest.testGetMaxColumnNameLength:

Unit test coverage change:

Integration test coverage change:

Uh oh!

wingman-pentaho commented May 6, 2016

Build Commands:

Build:

Cleanup:

Changed files: