Skip to content

feat: Add Db2 z/OS support#1721

Open
nj1973 wants to merge 120 commits intodevelopfrom
1630-db2-zos-support
Open

feat: Add Db2 z/OS support#1721
nj1973 wants to merge 120 commits intodevelopfrom
1630-db2-zos-support

Conversation

@nj1973
Copy link
Copy Markdown
Collaborator

@nj1973 nj1973 commented Apr 2, 2026

Description of changes

Adds support for Db2 z/OS and makes the distinction that our existing support was for Db2 LUW.

In the most part Db2 z/OS shares code with LUW. We've added a new Backend because there are some differences, but not a huge amount. The underlying ibm_db/ibm_db_sa drivers are used for both.

Notable differences:

  • Sorting by rand() is not supported, we needed a specific random row sample query that selects rand and sorts by the column alias.
  • LOB & XML data types are not as flexible on z/OS and needed extra restrictions.
  • Catalog SQL for data type detection was slightly different.
  • On Db2 z/OS we saw two hidden/auto generated columns being returned by the IBM driver, db2_generated_docid_for_xml and db2_generated_rowid_for_lob. We added code to exclude these from the Ibis tables because we could not influence the SQL used by the driver.
  • Db2 z/OS ROWID data type is unsupported and causes an exception. The customer who needs this support did not use ROWIDs explicitly and therefore this was not a blocker.
  • Generated SQL statements do not support bind variables. A few functions had to be changed to explicitly include tokens that would otherwise be changed to variables, for example the replacement text used with coalesce.

As a side task I moved some Impala/Hive code out of operations.py into dedicated files.

Unfortunately we have no access to a Db2 z/OS test system. For the most part I've been able to rely on testing code using LUW. I have also benefited from Db2 z/OS testing by the customer who requested this work.

Issues to be closed

Closes #1630

Checklist

  • I have followed the CONTRIBUTING Guide.
  • I have commented my code, particularly in hard-to-understand areas
  • I have updated any relevant documentation to reflect my changes, if applicable
  • I have added unit and/or integration tests relevant to my change as needed
  • I have already checked locally that all unit tests and linting are passing (use the tests/local_check.sh script)
  • I have manually executed end-to-end testing (E2E) with the affected databases/engines

@nj1973 nj1973 linked an issue Apr 2, 2026 that may be closed by this pull request
@nj1973 nj1973 marked this pull request as ready for review April 8, 2026 16:44
@nj1973
Copy link
Copy Markdown
Collaborator Author

nj1973 commented Apr 9, 2026

/gcbrun

@nj1973
Copy link
Copy Markdown
Collaborator Author

nj1973 commented Apr 10, 2026

/gcbrun

Comment thread third_party/ibis/ibis_db2/__init__.py Outdated
@@ -58,6 +61,7 @@ def do_connect(
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is a bad idea for us to set the default value for this port - this is true for all dbs. The underlying driver sets the default port, so we don't have to. If there is every a change in the default in the future, you need to update this.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's not actually a line I changed. Although I did copy that line into the z/OS version of the same method.

I've removed the port default from both implementations.

@sundar-mudupalli-work
Copy link
Copy Markdown
Collaborator

LGTM. Unfortunately, we can't test these change nightly.

@nj1973
Copy link
Copy Markdown
Collaborator Author

nj1973 commented Apr 13, 2026

/gcbrun

@nj1973
Copy link
Copy Markdown
Collaborator Author

nj1973 commented Apr 14, 2026

/gcbrun

@sundar-mudupalli-work
Copy link
Copy Markdown
Collaborator

/gcbrun

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Db2 z/OS support

2 participants