Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -255,7 +255,6 @@ The list below contains the functionality that contributors are planning to deve
* [x] [Offline Feature Server (alpha)](https://docs.feast.dev/reference/feature-servers/offline-feature-server)
* [x] [Registry server (alpha)](https://github.com/feast-dev/feast/blob/master/docs/reference/feature-servers/registry-server.md)
* **Data Quality Management (See [RFC](https://docs.google.com/document/d/110F72d4NTv80p35wDSONxhhPBqWRwbZXG4f9mNEMd98/edit))**
* [x] ~~Data profiling and validation (Great Expectations)~~ (deprecated)
* [x] [Feature Quality Monitoring](https://docs.feast.dev/how-to-guides/feature-monitoring) — built-in metrics, drift detection, serving log monitoring, and UI dashboard
* **Feature Discovery and Governance**
* [x] Python SDK for browsing feature registry
Expand Down
2 changes: 1 addition & 1 deletion docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ Feast helps ML platform/MLOps teams with DevOps experience productionize real-ti
* **batch feature engineering**: Feast supports on-demand and streaming transformations. Feast is also investing in supporting batch transformations.
* **native streaming feature integration:** Feast enables users to push streaming features, but does not pull from streaming sources or manage streaming pipelines.
* **lineage:** Feast helps tie feature values to model versions, but is not a complete solution for capturing end-to-end lineage from raw data sources to model versions. Feast also has community contributed plugins with [DataHub](https://datahubproject.io/docs/generated/ingestion/sources/feast/) and [Amundsen](https://github.com/amundsen-io/amundsen/blob/4a9d60176767c4d68d1cad5b093320ea22e26a49/databuilder/databuilder/extractor/feast\_extractor.py).
* **data quality / drift detection**: Feast now includes built-in [Feature Quality Monitoring](how-to-guides/feature-monitoring.md) that computes statistical metrics (null rates, distributions, percentiles), detects drift across batch data and serving logs, and provides a monitoring UI dashboard. The older Great Expectations integration is deprecated.
* **data quality / drift detection**: Feast includes built-in [Feature Quality Monitoring](how-to-guides/feature-monitoring.md) that computes statistical metrics (null rates, distributions, percentiles), detects drift across batch data and serving logs, and provides a monitoring UI dashboard.

## Example use cases

Expand Down
3 changes: 1 addition & 2 deletions docs/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,6 @@
* [Fraud detection on GCP](tutorials/tutorials-overview/fraud-detection.md)
* [Real-time credit scoring on AWS](tutorials/tutorials-overview/real-time-credit-scoring-on-aws.md)
* [Driver stats on Snowflake](tutorials/tutorials-overview/driver-stats-on-snowflake.md)
* [\[Deprecated\] Validating historical features with Great Expectations](tutorials/validating-historical-features.md)
* [Building streaming features](tutorials/building-streaming-features.md)
* [Retrieval Augmented Generation (RAG) with Feast](tutorials/rag-with-docling.md)
* [RAG Fine Tuning with Feast and Milvus](../examples/rag-retriever/README.md)
Expand Down Expand Up @@ -206,7 +205,7 @@
* [\[Beta\] On demand feature view](reference/beta-on-demand-feature-view.md)
* [\[Alpha\] Static Artifacts Loading](reference/alpha-static-artifacts.md)
* [\[Alpha\] Vector Database](reference/alpha-vector-database.md)
* [\[Deprecated\] Data quality monitoring (Great Expectations)](reference/dqm.md)
* [Data Quality Monitoring](reference/dqm.md)
* [\[Alpha\] Streaming feature computation with Denormalized](reference/denormalized.md)
* [\[Alpha\] Feature View Versioning](reference/alpha-feature-view-versioning.md)
* [OpenLineage Integration](reference/openlineage.md)
Expand Down
87 changes: 23 additions & 64 deletions docs/adr/ADR-0011-data-quality-monitoring.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Status

Accepted
Superseded — The original external-library-based validation has been replaced by Feast's native [Feature Quality Monitoring](../how-to-guides/feature-monitoring.md) system (`feast monitor run`).

## Context

Expand All @@ -12,92 +12,51 @@ Data quality issues can significantly impact ML model performance. Several compl
- **Upstream pipeline bugs**: Bugs in upstream pipelines can cause invalid values to overwrite existing valid values in an online store.
- **Training/serving skew**: Distribution shift between training and serving data can decrease model performance.

Feast needed a mechanism to validate data at retrieval time to catch these issues before they affect model training or serving.
Feast needed a mechanism to validate data to catch these issues before they affect model training or serving.

## Decision

Introduce a Data Quality Monitoring (DQM) module that validates datasets against user-curated rules, initially targeting historical retrieval (training dataset generation).
Introduce a Data Quality Monitoring (DQM) module that validates datasets against user-curated rules.

### Design
### Original Design (now replaced)

The validation process uses a **reference dataset** and a **profiler** pattern:
The original validation process used a **reference dataset** and a **profiler** pattern:

1. User prepares a reference dataset (saved from a known-good historical retrieval).
2. User defines a profiler function that produces a profile (set of expectations) from a dataset.
3. Validation is performed by comparing the tested dataset against the reference profile.

### Integration with Great Expectations
This approach was limited to historical retrieval only, required additional dependencies, and offered no built-in UI or automation.

The initial implementation uses [Great Expectations](https://greatexpectations.io/) as the validation engine:
### Current Design

```python
from feast.dqm.profilers.ge_profiler import ge_profiler
from great_expectations.dataset import Dataset
from great_expectations.core.expectation_suite import ExpectationSuite
The current system (`feast monitor run`) provides:

@ge_profiler
def my_profiler(dataset: Dataset) -> ExpectationSuite:
dataset.expect_column_max_to_be_between("column", 1, 2)
dataset.expect_column_values_to_not_be_null("important_feature")
return dataset.get_expectation_suite()
```

### Usage

Validation is triggered during historical feature retrieval via a `validation_reference` parameter:

```python
from feast import FeatureStore

store = FeatureStore(".")

job = store.get_historical_features(...)
df = job.to_df(
validation_reference=store
.get_saved_dataset("my_reference_dataset")
.as_reference(profiler=my_profiler)
)
```

If validation fails, a `ValidationFailed` exception is raised with details for all expectations that didn't pass. If validation succeeds, the materialized dataset is returned normally.

### Key Decisions
- Automatic metric computation (null rates, percentiles, histograms) with no external dependencies
- Monitoring across batch data and serving logs
- CLI and REST API for automation
- Built-in UI monitoring dashboard
- Support for all offline store backends via SQL push-down

- **Profiler-based approach**: Users define their own validation rules via profiler functions rather than Feast prescribing fixed validation rules.
- **Great Expectations integration**: Leverages an established data validation framework rather than building custom validation logic.
- **Validation at retrieval time**: Validation is performed when datasets are materialized (`.to_df()` or `.to_arrow()`), not during ingestion.
- **ValidationReference as a registry object**: Saved datasets and their validation references are stored in the Feast registry for reuse.
See [Feature Quality Monitoring](../how-to-guides/feature-monitoring.md) for full documentation.

## Consequences

### Positive

- Users can detect data quality issues before they affect model training.
- Flexible profiler pattern allows custom validation rules per use case.
- Integration with Great Expectations provides a rich set of built-in expectations.
- Reference datasets provide a baseline for detecting data drift.
- Native integration requires no extra dependencies.
- Covers both batch data and serving logs.
- Built-in UI provides immediate visibility into feature health.
- Baselines computed automatically on `feast apply`.

### Negative

- Currently limited to historical retrieval; online store write/read validation is planned but not yet implemented.
- Dependency on Great Expectations adds to the install footprint (optional via `feast[ge]`).
- Automatic profiling capabilities are limited; manual expectation crafting is recommended.

## Superseded

This ADR documents the original GE-based approach which is now **deprecated**. It has been superseded by Feast's built-in [Feature Quality Monitoring](../how-to-guides/feature-monitoring.md) system (introduced in 2025), which provides:

- Automatic metric computation (null rates, percentiles, histograms) with no external dependencies
- Monitoring across batch data and serving logs
- CLI (`feast monitor run`) and REST API for automation
- Built-in UI monitoring dashboard
- Support for all offline store backends via SQL push-down

The GE-based integration may be removed in a future release.
- Migration required from the original profiler-based approach.

## References

- Original RFC: Feast RFC-027: Data Quality Monitoring
- Implementation: `sdk/python/feast/dqm/`, `sdk/python/feast/saved_dataset.py`
- Documentation: [Data Quality Monitoring (deprecated)](../reference/dqm.md)
- **New system:** [Feature Quality Monitoring](../how-to-guides/feature-monitoring.md)
- Original RFC: Feast RFC-027: Data Quality Monitoring
- Implementation: `sdk/python/feast/monitoring/`
- Documentation: [Data Quality Monitoring](../reference/dqm.md)
- [Feature Quality Monitoring guide](../how-to-guides/feature-monitoring.md)
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ We are delighted to announce the release of Feast [0.18](https://github.com/feas

* Snowflake offline store, which allows you to define and use features stored in Snowflake.
* [Experimental] Saved Datasets, which allow training datasets to be persisted in an offline store.
* [Experimental] Data quality monitoring, which allows you to validate your training data with Great Expectations. Future work will allow you to detect issues with upstream data pipelines and check for training-serving skew.
* [Experimental] Data quality monitoring, which allows you to validate your training data. This has since been superseded by Feast's native [Feature Quality Monitoring](../how-to-guides/feature-monitoring.md) system.
* Python feature server graduation from alpha status.
* Performance improvements to on demand feature views, protobuf serialization and deserialization, and the Python feature server.

Expand All @@ -22,7 +22,7 @@ Training datasets generated via `get_historical_features` can now be persisted i

### [Experimental] Data quality monitoring

Feast 0.18 includes the first milestone of our data quality monitoring work. Many users have requested ways to validate their training and serving data, as well as monitor for training-serving skew. Feast 0.18 allows users to validate their training data through an integration with [Great Expectations](https://greatexpectations.io/). Users can declare one of the previously generated training datasets as a reference for this validation by persisting it as a "saved dataset" (see previous section). More details about future milestones of data quality monitoring can be found [here](https://docs.feastsite.wpenginepowered.com/v/master/reference/data-quality). There's also a [tutorial on validating historical features](https://docs.feastsite.wpenginepowered.com/v/master/how-to-guides/validation/validating-historical-features) that demonstrates all new concepts in action.
Feast 0.18 includes the first milestone of our data quality monitoring work. Many users have requested ways to validate their training and serving data, as well as monitor for training-serving skew. Feast 0.18 allows users to validate their training data by declaring previously generated training datasets as a reference for validation, persisted as "saved datasets" (see previous section). This initial integration has since been superseded by Feast's native [Feature Quality Monitoring](../how-to-guides/feature-monitoring.md) system, which provides built-in metrics computation, drift detection, serving log monitoring, and a UI dashboard.

### Performance improvements

Expand Down
2 changes: 1 addition & 1 deletion docs/getting-started/concepts/dataset.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# \[Alpha] Saved dataset

Feast datasets allow for conveniently saving dataframes that include both features and entities to be subsequently used for data analysis and model training. Data Quality Monitoring was the original motivation for creating the dataset concept. Note that the Great Expectations-based validation that used saved datasets is now deprecated in favor of Feast's built-in [Feature Quality Monitoring](../../how-to-guides/feature-monitoring.md) system, which does not require saved datasets.
Feast datasets allow for conveniently saving dataframes that include both features and entities to be subsequently used for data analysis and model training. Data Quality Monitoring was the original motivation for creating the dataset concept.

Dataset's metadata is stored in the Feast registry and raw data (features, entities, additional input keys and timestamp) is stored in the [offline store](../components/offline-store.md).

Expand Down
2 changes: 1 addition & 1 deletion docs/reference/codebase-structure.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ The majority of Feast logic lives in these Python files:

There are also several important submodules:
* `infra/` contains all the infrastructure components, such as the provider, offline store, online store, batch materialization engine, and registry.
* `dqm/` covers data quality monitoring. The legacy Great Expectations profiler (`profilers/ge_profiler`) is deprecated; see [`monitoring/`](../../sdk/python/feast/monitoring/) for the current built-in monitoring system.
* `dqm/` covers data quality monitoring. See [`monitoring/`](../../sdk/python/feast/monitoring/) for the built-in monitoring system.
* `diff/` covers the logic for determining how to apply infrastructure changes upon feature repo changes (e.g. the output of `feast plan` and `feast apply`).
* `embedded_go/` covers the Go feature server.
* `ui/` contains the embedded Web UI, to be launched on the `feast ui` command.
Expand Down
Loading
Loading