Dynamic Inference Headers with Prediction Trie Integration#1483

Merged

rapids-bot[bot] merged 41 commits intoNVIDIA:developfrom

dnandakumar-nv:dynamic-inference-headers

Jan 26, 2026

Commits on Jan 24, 2026

feat(profiler): add prediction trie data models
dnandakumar-nv
committed
feat(profiler): add MetricsAccumulator for prediction trie

dnandakumar-nv
and
claude
committed
feat(profiler): add PredictionTrieBuilder

dnandakumar-nv
and
claude
committed
feat(profiler): add PredictionTrieLookup
dnandakumar-nv
committed
feat(profiler): add prediction trie serialization

dnandakumar-nv
and
claude
committed
feat(llm): add LLMCallTracker for runtime prediction lookups

dnandakumar-nv
and
claude
committed
feat(profiler): integrate prediction trie generation

dnandakumar-nv
and
claude
committed
feat(llm): add prediction header injection to Dynamo client

dnandakumar-nv
and
claude
committed
feat(llm): add prediction_trie_path config to DynamoModelConfig

dnandakumar-nv
and
claude
committed
test(profiler): add end-to-end prediction trie test

dnandakumar-nv
and
claude
committed
docs: add runtime prediction trie integration design

dnandakumar-nv
and
claude
committed
docs: add runtime prediction trie implementation plan

dnandakumar-nv
and
claude
committed
feat(context): add function_path_stack ContextVar to ContextState

dnandakumar-nv
and
claude
committed
feat(context): update push_active_function to track function path stack

dnandakumar-nv
and
claude
committed
feat(step_manager): increment LLM call tracker on LLM_START events

dnandakumar-nv
and
claude
committed
Add dynamic prediction hook for runtime trie lookups

dnandakumar-nv
and
claude
committed
fix(test): include prediction_trie_path in dynamo field names test

dnandakumar-nv
and
claude
committed
Add prediction_lookup parameter to create_httpx_client_with_dynamo_hooks

dnandakumar-nv
and
claude
committed
Add end-to-end integration test for runtime prediction trie

dnandakumar-nv
and
claude
committed
docs: add prediction trie example config design

dnandakumar-nv
and
claude
committed
feat(examples): add prediction trie example configs and docs

dnandakumar-nv
and
claude
committed
Refactor header injection with dynamic prediction logic
dnandakumar-nv
committed
Refactor Dynamo prefix handling to centralize logic.
dnandakumar-nv
committed
Refactor DynamoPrefixContext for depth-aware prefix handling
dnandakumar-nv
committed
Refactor DynamoPrefixContext for depth-aware prefix handling
dnandakumar-nv
committed
Merge branch 'develop' into dynamic-inference-headers
dnandakumar-nv
authored
Refactor DynamoPrefixContext for depth-aware prefix handling
dnandakumar-nv
committed
Merge remote-tracking branch 'origin/dynamic-inference-headers' into dynamic-inference-headers
dnandakumar-nv
committed
Remove DynamoPrefixContext handling in Runner class
dnandakumar-nv
committed

Commits on Jan 26, 2026