This document outlines the roadmap for the Unity Catalog open source project. As always, features may move in/out of milestones pending available resources and priorities.
By more tightly integrating the already released credential and external locations API with the rest of the server internals, the next release will allow for more fine-grained, dynamic, and online management of storage locations and their credentials. Furthermore operators can delegate some storage management to the catalog via the managed locations for catalogs and schemas features.
Catalog managed commits are the basis for many new and powerful client (Delta) and server side features. Supporting the table scan and commit APIs is a key priority for the upcoming release.
OAuth support is important for cloud users and RBAC, and unity client plans to support common OAuth flows for authentication.
| Feature | Area | v0.1 | v0.2 | v0.3 | v0.4 | v0.5+ |
| Core | ||||||
| Catalog | API + Server |  |
|
|
|
|
| Schema | API + Server | |
|
|
|
|
| Managed location in catalog | API + Server | |
|
|||
| Managed location in schema | API + Server | |
|
|||
| Credential | API + Server | 🛠️ | |
|
||
| External Location | API + Server | 🛠️ | |
|
||
| Multi-tenancy | API + Server | |
||||
| Identity & Authentication | ||||||
| Local identity management (user) | API + Server | |
|
|
|
|
| Group management | API + Server | |
||||
| Support for Machine identities (SPs) | API + Server | |
||||
| SCIM to support identity sync from IdP (users and groups) | API + Server | |
|
|
|
|
| OAuth/OIDC for Users | API + Server | |
|
|
|
|
| OAuth/OIDC for Services | API + Server | |
||||
| OAuth client-side support | Spark integration | |
|
|||
| SAML authentication support | API + Server | ❓ | ||||
| Access Control & Governance | ||||||
| Support for change of ownership | API + Server | |
||||
| Add permission/privilege support for MODIFY, CREATE_X, BROWSE | API + Server | |
||||
| Add remaining permissions/privileges (MANAGE etc) | API + Server | |
||||
| Permission parity with Databricks UC | API + Server | |
||||
| Temporary credential vending for tables | API + Server | |
|
|
|
|
| Temporary credential vending for volumes | API + Server | |
|
|
|
|
| Temporary credential vending for models | API + Server | |
|
|
|
|
| Basic grants | API + Server | |
|
|
|
|
| Auditing | API + Server | |
||||
| SQL DCL changes | Spark Integration | |
||||
| RBAC | API + Server | ❓ | ||||
| Row level filters | API + Server | ❓ | ||||
| Column level masks | API + Server | ❓ | ||||
| ABAC | API + Server | ❓ | ||||
| Lineage | API + Server | ❓ | ||||
| Server production-readiness (support running as a HMS replacement) | ||||||
| Monitoring and Telemetry | API + Server | ❓ | ||||
| Database schema upgrades | API + Server | ❓ | ||||
| Change events | API + Server | ❓ | ||||
| Tables | ||||||
| External table reads & writes | API + Server | |
|
|
|
|
| Spark integration | |
|
|
|
||
| Delta integration | |
|
|
|
||
| Managed Delta table reads | API + Server | |
|
|
|
|
| Delta+Spark integration | |
|
|
|
|
|
| Managed Delta tables creates+writes with catalog-managed commits | API + Server | |
|
|||
| Delta-Spark integration | |
|
|
|||
| Delta Kernel integration | |
|
||||
| Delta Uniform tables with read as Iceberg via Iceberg REST API | API + Server | 🛠️ | 🛠️ | 🛠️ | 🛠️ | |
| Delta integration | |
|||||
| Iceberg tables with create+read+write | API + Server | |
||||
| Multi-engine data types for column definitions | API + Server | |
||||
| Views | ||||||
| Basic Spark SQL flavor views | API + Server | |
||||
| Multi-dialect views | API + Server | |
||||
| Iceberg view support | API + Server | |
||||
| Materialized views | API + Server | 🛠️ | |
|||
| Streaming tables | API + Server | 🛠️ | |
|||
| Shallow clones | API + Server | |
||||
| Non-tabular and AI assets | ||||||
| Functions (SQL UDFs, Python UDFs) | API + Server | |
|
|
|
|
| ML integrations with advanced python SDK | |
|
|
|||
| Spark integration | |
|||||
| Multi-engine functions (SQL) | API + Server | ❓ | ||||
| Remote functions | API + Server | ❓ | ||||
| External volumes | API + Server | |
|
|
|
|
| Spark integration | |
|||||
| Managed volumes | API + Server | |
||||
| Spark integration | |
|||||
| Models and model versions | API + Server | |
|
|
|
|
| MLflow integration | |
|
|
|||
| Spark integration | |
|||||
| Features tables | API + Server | ❓ | ||||
| Data monitors | API + Server | ❓ | ||||
| Sharing | ||||||
| Delta Sharing integration | API + Server | ❓ | ||||
| Shares | API + Server | ❓ | ||||
| Recipients | API + Server | ❓ | ||||
| Providers | API + Server | ❓ | ||||
| Federation | ||||||
| Connections | API + Server | ❓ | ||||
| Foreign objects (catalogs, schemas, tables) | API + Server | ❓ | ||||
| Support for different data sources: JDBC, Iceberg REST, HMS | API + Server | ❓ | ||||
| UI (needs to be completed) | ||||||