Always map java.sql.Types.BIGINT to LongType

[ngerstle]

java.sql.Types.BIGINT => LongType in all cases for Redshift.
Redshift has no unsigned BigInt, and thus BigInt maps to Long without loss of resolution.
See issue #310

[JoshRosen]

Thanks for the fix! Mind adding a regression test to an existing suite? If you can't, I can tackle it in a followup. Code from your original issue would be a good foundation for the test case. Check out RedshiftReadSuite, since that's where I'd add it.

…

On Fri, Dec 9, 2016 at 1:20 AM ngerstle ***@***.***> wrote: java.sql.Types.BIGINT => LongType in all cases for Redshift. Redshift has no unsigned BigInt, and thus BigInt maps to Long without loss of resolution. See issue #310 <#310> ------------------------------ You can view, comment on, or merge this pull request online at: #311 Commit Summary - Update RedshiftJDBCWrapper.scala File Changes - *M* src/main/scala/com/databricks/spark/redshift/RedshiftJDBCWrapper.scala <https://github.com/databricks/spark-redshift/pull/311/files#diff-0> (2) Patch Links: - https://github.com/databricks/spark-redshift/pull/311.patch - https://github.com/databricks/spark-redshift/pull/311.diff — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#311>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AADGPO5Ox0OGiqp5pwDLug4_usBaAuAbks5rGR1AgaJpZM4LIulr> .

[ngerstle]

I'm not able to find any clear point at which a decimal would be converted to a double, meaning that issue #310 is unlikely to fixed by this, but the dead code should still be pruned.
I added a couple of rough regression tests- please flesh out as recommended..

[codecov-io]

Current coverage is 88.55% (diff: 100%)

Merging #311 into master will not change coverage

@@             master       #311   diff @@
==========================================
  Files            15         15          
  Lines           769        769          
  Methods         614        615     +1   
  Messages          0          0          
  Branches        155        154     -1   
==========================================
  Hits            681        681          
  Misses           88         88          
  Partials          0          0          

Powered by Codecov. Last update 8adfe95...6a16c4b

[JoshRosen]

Regarding the double issue, is there a possibility that our call to ask Redshift the schema is returning a Double but then Redshift is actually turning around and unexpectedly giving us a Decimal? Do you have any test case which reproduces the Double behavior, even if you don't know how to actually fix the bug?

[ngerstle]

Unfortunately, I'm still trying to get a consistent abstract case- as mentioned in #310, I have a select count(column), count(distinct(column) from table that iterates over columns: sometimes it returns (bigint, bigint) and sometimes (double, bigint) (when debugging the schema of the dataframe returned when using the query option on sqlContext.read....