feat(openfeature): emit server-side EVP flagevaluation#8787
Draft
leoromanovsky wants to merge 21 commits into
Draft
feat(openfeature): emit server-side EVP flagevaluation#8787leoromanovsky wants to merge 21 commits into
leoromanovsky wants to merge 21 commits into
Conversation
…regation - Tests for canonical context key (type-tagged, not a hash, reviewer concern #3) - Tests for context pruning 256 fields/256 chars (reviewer concern #1) - Tests for two-tier aggregation: full tier, degraded overflow, drop-count (concern #8) - Tests for runtime_default from absent variant (reviewer concern #5) - Tests for EVP path constant and payload shape (NullValueHandling.Ignore, concern #2)
…sk 1) - FlagEvaluationApi with FlagEvaluationPath = "evp_proxy/v2/api/v2/flagevaluations" - FlagEvaluationAggregator: two-tier (full → degraded → drop-counted), NO ultra-degraded - Comparable canonical-context key (sorted, type-tagged length-delimited, NOT a hash) - Context pruning: 256 fields / 256 chars (reviewer concern #1) - Caps: globalCap=131072, perFlagCap=10000, degradedCap=32768 (concern #8) - runtime_default_used from absent variant (reviewer concern #5) - NullValueHandling.Ignore for optional-field omission per tier (concern #2) - AgentTransportStrategy + EventPlatformHeaderHelper, 10s SendInterval - 18 unit tests all passing
- FlagEvalEVPHookTests covers variant=null runtime_default_used, degraded tier omits context+targetingKey, allocationKey+targetingKey in full payload, empty aggregator returns null, EVP path constant, and FeatureFlagsModule.GetEVPApi() RED gate - FeatureFlagsModule_WhenEnabled_ExposesEVPApi fails (GetEVPApi not yet implemented)
…luation - Add FlagEvalEVPHook: FinallyAsync hook capturing all eval paths (success/error/default) per reviewer concern #7; routes through FeatureFlagsSdk.EnqueueEVP delegate bridge - Add OpenFeatureSdkEnqueueEVPIntegration: CallTarget instrumentation intercepting EnqueueEVP stub in Datadog.FeatureFlags.OpenFeature, routing to FeatureFlagsModule.GetEVPApi().Enqueue() - Update DatadogProvider: register FlagEvalEVPHook alongside existing FlagEvalMetricsHook when DD_FLAGGING_EVALUATION_COUNTS_ENABLED != "false" (dual killswitch) - Update FeatureFlagsModule: create FlagEvaluationApi when EVP enabled; expose GetEVPApi() for CallTarget bridge; dispose EVP api alongside exposure api - Update FeatureFlagsSdk: add no-op EnqueueEVP stub with NoInlining for CallTarget hook - Add DD_FLAGGING_EVALUATION_COUNTS_ENABLED to supported-configurations.yaml and all four generated ConfigurationKeys.FeatureFlags.g.cs TFM variants - Add DD_FLAGGING_EVALUATION_COUNTS_ENABLED to config_norm_rules.json to satisfy AllConfigurationValuesAreRegisteredWithIntake test
…tercepts the stub The native profiler's method hook table (generated_calltargets.g.cpp) was missing the EnqueueEVP entry for Datadog.FeatureFlags.OpenFeature.FeatureFlagsSdk. The CLR profiler only instruments methods listed in this table at JIT time; without the entry, FlagEvalEVPHook.FinallyAsync called the stub directly (a no-op), so nothing was ever enqueued and the EVP flagevaluation count stayed at zero. Also adds the corresponding entry to supported_calltargets.g.json for integration test consistency.
…hand-off Route the DD_FLAGGING_EVALUATION_COUNTS_ENABLED killswitch through TracerSettings (parsed, telemetry-reported, overridable) instead of a raw environment read, and register the EVP hook unconditionally in the provider so the killswitch lives on the tracer side. Add an async hand-off: the hot-path Enqueue does a cheap bounded ConcurrentQueue offer with backpressure drop counting, and the background send loop drains into the two-tier aggregator and drains again on shutdown so a process exit does not lose the last window. Stamp the evaluation time (dd.eval.timestamp_ms) at provider entry so first/last_evaluation reflect the evaluation moment. Fix the batch wrapper to serialize as camelCase "flagEvaluations" while inner event fields stay snake_case. Add a wired BenchmarkDotNet hot-path benchmark and expand unit coverage for the aggregation, payload, transport, backpressure, and shutdown-drain contracts.
This comment has been minimized.
This comment has been minimized.
BenchmarksBenchmark execution time: 2026-06-28 22:49:20 Comparing candidate commit 68417d8 in PR branch Found 0 performance improvements and 3 performance regressions! Performance is the same for 69 metrics, 0 unstable metrics, 63 known flaky benchmarks, 63 flaky benchmarks without significant changes.
|
Execution-Time Benchmarks Report ⏱️Execution-time results for samples comparing This PR (8787) and master. ✅ No regressions detected - check the details below Full Metrics ComparisonFakeDbCommand
HttpMessageHandler
Comparison explanationExecution-time benchmarks measure the whole time it takes to execute a program, and are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are highlighted in **red**. The following thresholds were used for comparing the execution times:
Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard. Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph). Duration chartsFakeDbCommand (.NET Framework 4.8)gantt
title Execution time (ms) FakeDbCommand (.NET Framework 4.8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8787) - mean (73ms) : 68, 77
master - mean (73ms) : 69, 77
section Bailout
This PR (8787) - mean (74ms) : 73, 76
master - mean (74ms) : 73, 76
section CallTarget+Inlining+NGEN
This PR (8787) - mean (1,082ms) : 1040, 1124
master - mean (1,082ms) : 1038, 1126
FakeDbCommand (.NET Core 3.1)gantt
title Execution time (ms) FakeDbCommand (.NET Core 3.1)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8787) - mean (113ms) : 108, 119
master - mean (114ms) : 106, 121
section Bailout
This PR (8787) - mean (111ms) : 109, 113
master - mean (111ms) : 109, 113
section CallTarget+Inlining+NGEN
This PR (8787) - mean (780ms) : 752, 809
master - mean (773ms) : 755, 791
FakeDbCommand (.NET 6)gantt
title Execution time (ms) FakeDbCommand (.NET 6)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8787) - mean (99ms) : 94, 104
master - mean (102ms) : 96, 108
section Bailout
This PR (8787) - mean (98ms) : 96, 99
master - mean (100ms) : 95, 105
section CallTarget+Inlining+NGEN
This PR (8787) - mean (939ms) : 896, 981
master - mean (937ms) : 891, 984
FakeDbCommand (.NET 8)gantt
title Execution time (ms) FakeDbCommand (.NET 8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8787) - mean (96ms) : 92, 100
master - mean (95ms) : 92, 98
section Bailout
This PR (8787) - mean (100ms) : 96, 105
master - mean (101ms) : 96, 106
section CallTarget+Inlining+NGEN
This PR (8787) - mean (810ms) : 774, 847
master - mean (815ms) : 772, 857
HttpMessageHandler (.NET Framework 4.8)gantt
title Execution time (ms) HttpMessageHandler (.NET Framework 4.8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8787) - mean (204ms) : 200, 208
master - mean (202ms) : 197, 206
section Bailout
This PR (8787) - mean (208ms) : 204, 211
master - mean (205ms) : 202, 209
section CallTarget+Inlining+NGEN
This PR (8787) - mean (1,215ms) : 1160, 1270
master - mean (1,206ms) : 1165, 1247
HttpMessageHandler (.NET Core 3.1)gantt
title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8787) - mean (292ms) : 284, 299
master - mean (291ms) : 285, 297
section Bailout
This PR (8787) - mean (292ms) : 287, 297
master - mean (292ms) : 288, 296
section CallTarget+Inlining+NGEN
This PR (8787) - mean (972ms) : 948, 996
master - mean (969ms) : 949, 989
HttpMessageHandler (.NET 6)gantt
title Execution time (ms) HttpMessageHandler (.NET 6)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8787) - mean (285ms) : 277, 293
master - mean (284ms) : 275, 292
section Bailout
This PR (8787) - mean (286ms) : 281, 292
master - mean (284ms) : 278, 290
section CallTarget+Inlining+NGEN
This PR (8787) - mean (1,168ms) : 1130, 1206
master - mean (1,177ms) : 1140, 1214
HttpMessageHandler (.NET 8)gantt
title Execution time (ms) HttpMessageHandler (.NET 8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8787) - mean (282ms) : 276, 288
master - mean (280ms) : 275, 285
section Bailout
This PR (8787) - mean (283ms) : 278, 288
master - mean (281ms) : 277, 285
section CallTarget+Inlining+NGEN
This PR (8787) - mean (1,051ms) : 999, 1103
master - mean (1,041ms) : 994, 1088
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
…2446-evp-flagevaluation-dotnet # Conflicts: # tracer/src/Datadog.Tracer.Native/Generated/generated_calltargets.g.cpp
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Customers need .NET services to produce the same server-side feature-flag evaluation signal as the other supported runtimes, and APM needs backend-visible evidence to approve the rollout. This contribution adds bounded EVP
flagevaluationdelivery alongside the existing OTelfeature_flag.evaluationsmetric path so .NET participates in the cross-SDK customer value without changing application evaluation behavior.Changes
flagevaluationwriter behindDD_FLAGGING_EVALUATION_COUNTS_ENABLED; OTel metrics remain unchanged.FlagEvalEVPHookso it is distinct fromFlagEvalMetricsHook.targeting_keyandcontextafter the per-flag full-fidelity cap is exceeded.targeting_keyandcontext; drops only if the degraded row still cannot fit.reason, runtime-default state,error.message, degraded omission, visible-dimension buckets, byte-size splitting, and oversized-row degradation/drop.flageval-workerschema fixture from the SDK test tree and keeps contract coverage as direct payload assertions.Decisions
flagevaluationdelivery is a separate bounded logging path from OTel metrics so this rollout does not change existing metric semantics or application evaluation behavior.reasonis intentionally excluded from payloads and cardinality.targeting_rule.keyis reserved for real targeting-rule metadata and is omitted when the provider does not supply it.{context, flagEvaluations}wrapper. Compression remains a wire concern after the 5 MiB decision.flowchart TD A[drained aggregated rows] --> B[serialize candidate batch as JSON] B --> C{batch <= 5 MiB?} C -- yes --> D[gzip and post asynchronously through EVP proxy] C -- no --> E{current row fits degraded?} E -- yes --> F[omit targeting_key and context] F --> B E -- no --> G[drop, log, count]Validation Evidence
Dogfooding App
ffe-dogfoodingapp-dotnetwas run with locally built .NET artifacts from this PR:Datadog.FeatureFlags.OpenFeature 2.3.0Datadog.Trace.Bundle 3.47.0PROVIDER_READY.DD_EVP_PROXY_CONFIG_ADDITIONAL_ENDPOINTS={},DD_SKIP_SSL_VALIDATION=false, andDD_REMOTE_CONFIGURATION_NO_TLS_VALIDATION=false.ffe-dogfooding-string-flag:dotnet-batch-evp-agent-20260623T030818Z-alphadotnet-batch-evp-agent-20260623T030818Z-bravodotnet-batch-evp-agent-20260623T030818Z-charlievariant_2.System Tests
Staging End-To-End
--datacenter us1.staging.dog --customer-auth=skipagainst theflagevaluationtrack for the exact targeting keys above.flag.key=ffe-dogfooding-string-flag,variant.key=variant_2,allocation.key=allocation-override-392dd7c149f8, andevaluation_count=5:dotnet-batch-evp-agent-20260623T030818Z-alpha:first_evaluation=1782184098608,last_evaluation=1782184098650,timestamp=1782184098747,evaluation_count=5dotnet-batch-evp-agent-20260623T030818Z-bravo:first_evaluation=1782184098660,last_evaluation=1782184098698,timestamp=1782184098747,evaluation_count=5dotnet-batch-evp-agent-20260623T030818Z-charlie:first_evaluation=1782184098708,last_evaluation=1782184098745,timestamp=1782184098747,evaluation_count=5