feat: Add remote materialization support via feature server#6590
Open
aniketpalu wants to merge 4 commits into
Open
feat: Add remote materialization support via feature server#6590aniketpalu wants to merge 4 commits into
aniketpalu wants to merge 4 commits into
Conversation
Allow store.materialize() and materialize_incremental() to delegate execution to the feature server instead of running the batch engine locally. This eliminates the need for engine-specific dependencies (PySpark, kubernetes client, etc.) and RBAC on the client pod. Components: - New shared poll_materialization_status() function - Client-side _remote_materialize() with registry-based FV state polling - POST /materialize-async and /materialize-incremental-async endpoints (202) - materialize_mode, url, timeout, poll_interval config fields - 19 unit tests + 2 E2E integration tests Addresses feast-dev#4526 (remote materialization). Signed-off-by: Aniket Paluskar <apaluska@redhat.com>
Signed-off-by: Aniket Paluskar <apaluska@redhat.com>
Allow store.materialize() and materialize_incremental() to delegate execution to the feature server instead of running the batch engine locally. This eliminates the need for engine-specific dependencies (PySpark, kubernetes client, etc.) and RBAC on the client pod. Components: - Shared poll_materialization_status() with stateful FV state tracking - Client-side _remote_materialize_common() with auth forwarding - POST /materialize-async and /materialize-incremental-async (202) - _force_local guard prevents infinite recursion on server - Concurrency guard returns 409 if FV already materializing - version parameter forwarded through full chain - materialize_mode, url, timeout, poll_interval config fields Addresses feast-dev#4526 (remote materialization). Signed-off-by: Aniket Paluskar <apaluska@redhat.com>
- Use allow_cache=False in concurrency guard to avoid stale registry reads - Close HTTP session in finally block to prevent resource leaks - Wrap remote HTTP call with descriptive error message and exception chaining - Add from-e chaining on timestamp parse errors - Improve timeout error message with actionable guidance - Add @model_validator to reject remote mode without url at config time - Add unit test for config validation Signed-off-by: Aniket Paluskar <apaluska@redhat.com>
jyejare
reviewed
Jul 9, 2026
Comment on lines
+15
to
+17
| materialize_mode: Literal["local", "remote"] = "local" | ||
| """When 'remote', store.materialize() delegates to the feature server's | ||
| async endpoint instead of running the batch engine locally.""" |
Collaborator
There was a problem hiding this comment.
I think we dont need to provide the materialize_mode config. We should strictly feature server take care of materialization delegation to compute engine. Leaving no choice to do materialization on client side.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
Allow store.materialize() and materialize_incremental() to delegate execution to the feature server instead of running the batch engine locally. This eliminates the need for engine-specific dependencies (PySpark, kubernetes client, etc.) and RBAC on the client pod. Components:
Which issue(s) this PR fixes:
Checks
git commit -s)Testing Strategy
Misc