Skip to content

Comments

Add ExperimentFramework.Diagnostics package with event sinks for standardized observability#59

Merged
JerrettDavis merged 6 commits intomainfrom
copilot/add-experiment-framework-diagnostics
Jan 15, 2026
Merged

Add ExperimentFramework.Diagnostics package with event sinks for standardized observability#59
JerrettDavis merged 6 commits intomainfrom
copilot/add-experiment-framework-diagnostics

Conversation

Copy link
Contributor

Copilot AI commented Jan 15, 2026

Description

Adds ExperimentFramework.Diagnostics package providing standardized event capture for experiment lifecycle events. Enables consistent logging, test assertions, and telemetry without bespoke per-app implementations.

Type of Change

  • New feature (non-breaking change which adds functionality)
  • Documentation update

Related Issues

Changes Made

Core Event Model

  • ExperimentEvent discriminated union with 7 event kinds: TrialStarted, TrialEnded, RouteSelected, FallbackOccurred, ExceptionThrown, MethodInvoked, MethodCompleted
  • IExperimentEventSink interface with OnEvent(in ExperimentEvent e) for pass-by-reference efficiency
  • ExperimentEventPublisher bridge component with factory methods for framework integration

Built-in Sinks

  • InMemoryExperimentEventSink: Bounded (ring buffer) or unbounded storage, thread-safe with ConcurrentQueue, query API for test assertions
  • LoggerExperimentEventSink: Structured logging with event IDs 1001-1007, appropriate log levels (Debug/Info/Warning/Error/Trace)
  • OpenTelemetryExperimentEventSink: Metrics (counters/histograms) for all events, activities for significant events (fallbacks/exceptions), BCL-only (System.Diagnostics)
  • CompositeExperimentEventSink: Deterministic ordering, smart exception handling (filters critical exceptions, provides debug output for troubleshooting)

DI Extensions

  • AddInMemoryExperimentEventSink(maxCapacity?), AddLoggerExperimentEventSink(categoryName?), AddOpenTelemetryExperimentEventSink()
  • GetExperimentEventSinks() returns composite of all registered sinks

Documentation

  • User guide: docs/user-guide/diagnostics.md with examples, performance notes, best practices
  • Package README and main README updated
  • Correct GitHub repository URLs (JerrettDavis/ExperimentFramework)

Code Quality Improvements

  • Enhanced exception handling in CompositeExperimentEventSink to filter critical exceptions (OutOfMemoryException, StackOverflowException) while providing debug output for other failures
  • Standardized test dependencies (xunit.runner.visualstudio 3.1.4) for consistency across test projects
  • Added type safety checks in tests for improved test reliability

Example Usage

// Registration
services.AddInMemoryExperimentEventSink(maxCapacity: 1000);
services.AddLoggerExperimentEventSink();
services.AddOpenTelemetryExperimentEventSink();

// Testing
[Fact]
public async Task Experiment_RecordsFallback()
{
    var sink = provider.GetRequiredService<InMemoryExperimentEventSink>();
    await service.ExecuteAsync();
    
    var fallbacks = sink.GetEventsByKind(ExperimentEventKind.FallbackOccurred);
    Assert.Single(fallbacks);
    Assert.Equal("experimental", fallbacks[0].TrialKey);
    Assert.Equal("control", fallbacks[0].FallbackKey);
}

Testing

  • Unit tests pass locally
  • All existing tests pass
  • Added new tests for new functionality

Test Configuration

  • OS: Linux (Ubuntu)
  • .NET Version: 10.0.101
  • Tests: 40 new tests covering all sinks, bounded/unbounded storage, thread-safety, event filtering, DI registration, composite sinks

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

Additional Context

Performance considerations:

  • Events passed by reference (in parameter) to avoid allocations
  • Bounded sinks use ring buffers to limit memory
  • OTel sink selectively emits activities only for significant events
  • No allocations in hot path for event forwarding

Exception handling strategy:

  • CompositeExperimentEventSink filters critical exceptions (OutOfMemoryException, StackOverflowException) to allow them to propagate
  • Non-critical exceptions from individual sinks are caught to prevent cascading failures
  • Debug.WriteLine provides troubleshooting information without impacting production performance

Acceptance criteria met:

  • ✅ Minimal allocations (pass-by-reference)
  • ✅ Deterministic sink ordering (registration order)
  • ✅ Bounded/unbounded in-memory storage
  • ✅ Structured logs with event IDs
  • ✅ OTel activities and metrics
Original prompt

This section details on the original issue you should resolve

<issue_title>[Feature] Add ExperimentFramework.Diagnostics package with first-class event sinks and in-memory/logging/OTel bridges</issue_title>
<issue_description>### Problem

ExperimentFramework supports decorators and logging, but event capture is not standardized. Teams implement bespoke logging/telemetry per app and per experiment.

A standardized event model enables:

  • consistent logging
  • consistent test assertions
  • powering simulation reports

Goals

  • Define stable ExperimentEvent types and sink contracts.

  • Provide built-in sinks:

    • InMemoryExperimentEventSink (for tests)
    • LoggerExperimentEventSink (ILogger)
    • OpenTelemetryExperimentEventSink (when OTel package present)

Non-goals

  • Not a full observability platform.

Proposed API Surface

  • IExperimentEventSink with OnEvent(in ExperimentEvent e)

  • ExperimentEvent (discriminated union style) with event kinds:

    • TrialStarted / TrialEnded
    • RouteSelected
    • FallbackOccurred
    • ExceptionThrown
    • (optional) MethodInvoked / MethodCompleted

Acceptance Criteria

  • Core can publish events without allocating excessively.
  • Sink ordering is deterministic.
  • In-memory sink supports bounded storage (ring buffer) and unbounded (test-only).
  • ILogger sink produces structured logs with event ids.
  • Optional OTel sink emits activities/metrics.
  • Docs: “Diagnostics & Tracing” with examples.

</issue_description>

Comments on the Issue (you are @copilot in this section)


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI and others added 3 commits January 15, 2026 04:03
Co-authored-by: JerrettDavis <2610199+JerrettDavis@users.noreply.github.com>
Co-authored-by: JerrettDavis <2610199+JerrettDavis@users.noreply.github.com>
Co-authored-by: JerrettDavis <2610199+JerrettDavis@users.noreply.github.com>
Copilot AI changed the title [WIP] Add ExperimentFramework.Diagnostics package with event sinks Add ExperimentFramework.Diagnostics package with event sinks for standardized observability Jan 15, 2026
Copilot AI requested a review from JerrettDavis January 15, 2026 04:14
@JerrettDavis JerrettDavis requested a review from Copilot January 15, 2026 04:45
@JerrettDavis JerrettDavis marked this pull request as ready for review January 15, 2026 04:50
@github-actions github-actions bot added documentation Improvements or additions to documentation area: tests dependencies labels Jan 15, 2026
@github-actions
Copy link

github-actions bot commented Jan 15, 2026

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

OpenSSF Scorecard

PackageVersionScoreDetails
nuget/Microsoft.Extensions.DependencyInjection.Abstractions 🟢 6.9
Details
CheckScoreReason
Maintained🟢 1030 commit(s) and 18 issue activity found in the last 90 days -- score normalized to 10
CII-Best-Practices⚠️ 0no effort to earn an OpenSSF best practices badge detected
Code-Review🟢 10all changesets reviewed
Packaging⚠️ -1packaging workflow not detected
Security-Policy🟢 10security policy file detected
Dangerous-Workflow🟢 10no dangerous workflow patterns detected
License🟢 10license file detected
Token-Permissions⚠️ 0detected GitHub workflow tokens with excessive permissions
Branch-Protection🟢 5branch protection is not maximal on development and all release branches
Signed-Releases⚠️ -1no releases found
Binary-Artifacts🟢 10no binaries found in the repo
SAST⚠️ 0SAST tool is not run on all commits -- score normalized to 0
Fuzzing⚠️ 0project is not fuzzed
Pinned-Dependencies🟢 9dependency not pinned by hash detected -- score normalized to 9
Vulnerabilities🟢 91 existing vulnerabilities detected
nuget/Microsoft.Extensions.Logging.Abstractions 🟢 6.9
Details
CheckScoreReason
Maintained🟢 1030 commit(s) and 18 issue activity found in the last 90 days -- score normalized to 10
CII-Best-Practices⚠️ 0no effort to earn an OpenSSF best practices badge detected
Code-Review🟢 10all changesets reviewed
Packaging⚠️ -1packaging workflow not detected
Security-Policy🟢 10security policy file detected
Dangerous-Workflow🟢 10no dangerous workflow patterns detected
License🟢 10license file detected
Token-Permissions⚠️ 0detected GitHub workflow tokens with excessive permissions
Branch-Protection🟢 5branch protection is not maximal on development and all release branches
Signed-Releases⚠️ -1no releases found
Binary-Artifacts🟢 10no binaries found in the repo
SAST⚠️ 0SAST tool is not run on all commits -- score normalized to 0
Fuzzing⚠️ 0project is not fuzzed
Pinned-Dependencies🟢 9dependency not pinned by hash detected -- score normalized to 9
Vulnerabilities🟢 91 existing vulnerabilities detected
nuget/Microsoft.Extensions.DependencyInjection 10.0.1 UnknownUnknown
nuget/Microsoft.Extensions.Logging 10.0.1 UnknownUnknown
nuget/Microsoft.NET.Test.Sdk 17.14.1 🟢 5
Details
CheckScoreReason
Code-Review🟢 6Found 12/18 approved changesets -- score normalized to 6
Maintained🟢 1025 commit(s) and 21 issue activity found in the last 90 days -- score normalized to 10
Packaging⚠️ -1packaging workflow not detected
CII-Best-Practices⚠️ 0no effort to earn an OpenSSF best practices badge detected
Dangerous-Workflow🟢 10no dangerous workflow patterns detected
License🟢 10license file detected
Security-Policy🟢 10security policy file detected
Token-Permissions⚠️ 0detected GitHub workflow tokens with excessive permissions
Branch-Protection⚠️ -1internal error: error during branchesHandler.setup: internal error: some github tokens can't read classic branch protection rules: https://github.com/ossf/scorecard-action/blob/main/docs/authentication/fine-grained-auth-token.md
Signed-Releases⚠️ -1no releases found
Binary-Artifacts⚠️ 0binaries present in source code
SAST⚠️ 0SAST tool is not run on all commits -- score normalized to 0
Pinned-Dependencies⚠️ 0dependency not pinned by hash detected -- score normalized to 0
Fuzzing⚠️ 0project is not fuzzed
Vulnerabilities🟢 91 existing vulnerabilities detected
nuget/coverlet.collector 6.0.4 🟢 5.6
Details
CheckScoreReason
Code-Review⚠️ 2Found 7/30 approved changesets -- score normalized to 2
Maintained🟢 1023 commit(s) and 17 issue activity found in the last 90 days -- score normalized to 10
Packaging⚠️ -1packaging workflow not detected
Dangerous-Workflow🟢 10no dangerous workflow patterns detected
CII-Best-Practices⚠️ 0no effort to earn an OpenSSF best practices badge detected
Token-Permissions🟢 9detected GitHub workflow tokens with excessive permissions
Binary-Artifacts🟢 7binaries present in source code
License🟢 10license file detected
Fuzzing⚠️ 0project is not fuzzed
Signed-Releases⚠️ 0Project has not signed or included provenance with any releases.
Branch-Protection⚠️ -1internal error: error during branchesHandler.setup: internal error: some github tokens can't read classic branch protection rules: https://github.com/ossf/scorecard-action/blob/main/docs/authentication/fine-grained-auth-token.md
Pinned-Dependencies⚠️ 0dependency not pinned by hash detected -- score normalized to 0
Security-Policy⚠️ 0security policy file not detected
Vulnerabilities🟢 100 existing vulnerabilities detected
SAST🟢 8SAST tool detected but not run on all commits
nuget/xunit 2.9.3 🟢 4.9
Details
CheckScoreReason
Dangerous-Workflow🟢 10no dangerous workflow patterns detected
Maintained🟢 1030 commit(s) and 27 issue activity found in the last 90 days -- score normalized to 10
Code-Review⚠️ 1Found 3/30 approved changesets -- score normalized to 1
CII-Best-Practices⚠️ 0no effort to earn an OpenSSF best practices badge detected
License🟢 9license file detected
Token-Permissions⚠️ 0detected GitHub workflow tokens with excessive permissions
Packaging⚠️ -1packaging workflow not detected
Branch-Protection⚠️ -1internal error: error during branchesHandler.setup: internal error: some github tokens can't read classic branch protection rules: https://github.com/ossf/scorecard-action/blob/main/docs/authentication/fine-grained-auth-token.md
Signed-Releases⚠️ -1no releases found
Security-Policy⚠️ 0security policy file not detected
Binary-Artifacts🟢 10no binaries found in the repo
Pinned-Dependencies⚠️ 0dependency not pinned by hash detected -- score normalized to 0
Fuzzing⚠️ 0project is not fuzzed
SAST⚠️ 0SAST tool is not run on all commits -- score normalized to 0
Vulnerabilities🟢 100 existing vulnerabilities detected
nuget/xunit.runner.visualstudio 3.1.4 UnknownUnknown

Scanned Files

  • src/ExperimentFramework.Diagnostics/ExperimentFramework.Diagnostics.csproj
  • tests/ExperimentFramework.Diagnostics.Tests/ExperimentFramework.Diagnostics.Tests.csproj

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds the ExperimentFramework.Diagnostics package to provide standardized event capture infrastructure for experiment lifecycle events. It enables consistent logging, test assertions, and telemetry without requiring bespoke implementations per application.

Changes:

  • Introduces core event model with 7 event kinds and an IExperimentEventSink interface
  • Provides three built-in sinks: InMemory (for testing), Logger (structured logging), and OpenTelemetry (metrics/activities)
  • Adds DI extension methods for easy registration and composition of multiple sinks
  • Includes comprehensive documentation and 40+ unit tests

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
src/ExperimentFramework.Diagnostics/ExperimentEvent.cs Core event model with discriminated union pattern for 7 event types
src/ExperimentFramework.Diagnostics/IExperimentEventSink.cs Interface definition with pass-by-reference efficiency
src/ExperimentFramework.Diagnostics/InMemoryExperimentEventSink.cs Thread-safe in-memory sink with bounded/unbounded storage
src/ExperimentFramework.Diagnostics/LoggerExperimentEventSink.cs Structured logging sink with event IDs 1001-1007
src/ExperimentFramework.Diagnostics/OpenTelemetryExperimentEventSink.cs OpenTelemetry metrics and activities using BCL types
src/ExperimentFramework.Diagnostics/CompositeExperimentEventSink.cs Composite pattern with deterministic ordering and exception isolation
src/ExperimentFramework.Diagnostics/ExperimentEventPublisher.cs Bridge component with factory methods for event creation
src/ExperimentFramework.Diagnostics/ExperimentDiagnosticsExtensions.cs DI extension methods for sink registration
src/ExperimentFramework.Diagnostics/ExperimentFramework.Diagnostics.csproj Multi-target package configuration (net8.0, net9.0, net10.0)
tests/ExperimentFramework.Diagnostics.Tests/* Comprehensive test coverage with 40+ tests for all sinks and scenarios
docs/user-guide/diagnostics.md Complete user guide with examples and best practices
README.md Updated main README with diagnostics section and example

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@codecov-commenter
Copy link

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (main@39d101a). Learn more about missing BASE report.

Additional details and impacted files
@@           Coverage Diff           @@
##             main      #59   +/-   ##
=======================================
  Coverage        ?   83.23%           
=======================================
  Files           ?      184           
  Lines           ?     6860           
  Branches        ?      962           
=======================================
  Hits            ?     5710           
  Misses          ?     1150           
  Partials        ?        0           
Flag Coverage Δ
unittests 83.23% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@JerrettDavis
Copy link
Owner

@copilot apply changes based on the comments in this thread

@JerrettDavis
Copy link
Owner

@copilot Retry

…ling, standardize xunit version, add type check

Co-authored-by: JerrettDavis <2610199+JerrettDavis@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Jan 15, 2026

@copilot Retry

Applied all code review feedback in commit b6b9354:

  1. GitHub URLs: Updated from yourusername to JerrettDavis in ExperimentFramework.Diagnostics.csproj
  2. Exception handling: Enhanced CompositeExperimentEventSink to filter critical exceptions (OutOfMemoryException, StackOverflowException) and added Debug.WriteLine for troubleshooting
  3. xunit version: Standardized to 3.1.4 to match other test projects
  4. Dictionary cast: Added Assert.IsType check before cast for safer test assertion

All 40 tests still passing.

…entSinkTests.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@github-actions
Copy link

Code Coverage

Summary
  Generated on: 01/15/2026 - 19:40:50
  Coverage date: 01/15/2026 - 19:40:02 - 01/15/2026 - 19:40:33
  Parser: MultiReport (2x Cobertura)
  Assemblies: 19
  Classes: 215
  Files: 185
  Line coverage: 80.2%
  Covered lines: 5539
  Uncovered lines: 1359
  Coverable lines: 6898
  Total lines: 26073
  Branch coverage: 72% (2180 of 3026)
  Covered branches: 2180
  Total branches: 3026
  Method coverage: 88.4% (881 of 996)
  Full method coverage: 76.9% (766 of 996)
  Covered methods: 881
  Fully covered methods: 766
  Total methods: 996

ExperimentFramework                                                                                  90.4%
  ExperimentFramework.Activation.ActivationEvaluator                                                 93.1%
  ExperimentFramework.Activation.SystemTimeProvider                                                   100%
  ExperimentFramework.Decorators.BenchmarkDecoratorFactory                                            100%
  ExperimentFramework.Decorators.DecoratorPipeline                                                    100%
  ExperimentFramework.Decorators.ErrorLoggingDecoratorFactory                                         100%
  ExperimentFramework.Decorators.TimeoutDecoratorFactory                                              100%
  ExperimentFramework.ExperimentBuilder                                                              81.1%
  ExperimentFramework.ExperimentBuilderExtensions                                                     100%
  ExperimentFramework.ExperimentFrameworkBuilder                                                      100%
  ExperimentFramework.ExperimentLoggingBuilder                                                        100%
  ExperimentFramework.ExperimentRegistry                                                              100%
  ExperimentFramework.KillSwitch.ExperimentDisabledException                                          100%
  ExperimentFramework.KillSwitch.InMemoryKillSwitchProvider                                           100%
  ExperimentFramework.KillSwitch.KillSwitchDecoratorFactory                                           100%
  ExperimentFramework.KillSwitch.NoopKillSwitchProvider                                               100%
  ExperimentFramework.KillSwitch.TrialDisabledException                                               100%
  ExperimentFramework.Metrics.MetricsDecoratorFactory                                                 100%
  ExperimentFramework.Metrics.NoopExperimentMetrics                                                   100%
  ExperimentFramework.Models.BehaviorRule                                                             100%
  ExperimentFramework.Models.Experiment                                                               100%
  ExperimentFramework.Models.ExperimentRegistration                                                    75%
  ExperimentFramework.Models.SelectionModeExtensions                                                  100%
  ExperimentFramework.Models.SelectionRule                                                            100%
  ExperimentFramework.Models.ServiceExperimentDefinition<T>                                           100%
  ExperimentFramework.Models.Trial                                                                    100%
  ExperimentFramework.Naming.DefaultExperimentNamingConvention                                        100%
  ExperimentFramework.Naming.ExperimentSelectorName                                                   100%
  ExperimentFramework.RuntimeExperimentProxy<T>                                                      81.8%
  ExperimentFramework.Selection.Providers.BooleanFeatureFlagProvider                                  100%
  ExperimentFramework.Selection.Providers.BooleanFeatureFlagProviderFactory                           100%
  ExperimentFramework.Selection.Providers.ConfigurationValueProvider                                   80%
  ExperimentFramework.Selection.Providers.ConfigurationValueProviderFactory                           100%
  ExperimentFramework.Selection.SelectionModeAttribute                                                100%
  ExperimentFramework.Selection.SelectionModeProviderBase                                             100%
  ExperimentFramework.Selection.SelectionModeProviderFactory<T>                                       100%
  ExperimentFramework.Selection.SelectionModeRegistry                                                 100%
  ExperimentFramework.ServiceCollectionExtensions                                                    77.6%
  ExperimentFramework.ServiceExperimentBuilder<T>                                                    94.5%
  ExperimentFramework.ServiceRegistration.OperationMetadata                                           100%
  ExperimentFramework.ServiceRegistration.OperationResult                                             100%
  ExperimentFramework.ServiceRegistration.PlanExecutionResult                                        83.3%
  ExperimentFramework.ServiceRegistration.RegistrationPlan                                            100%
  ExperimentFramework.ServiceRegistration.RegistrationPlanBuilder                                    97.2%
  ExperimentFramework.ServiceRegistration.RegistrationPlanExecutor                                   62.7%
  ExperimentFramework.ServiceRegistration.RegistrationPlanReport                                     98.1%
  ExperimentFramework.ServiceRegistration.ServiceGraphPatchOperation                                   77%
  ExperimentFramework.ServiceRegistration.ServiceGraphSnapshot                                        100%
  ExperimentFramework.ServiceRegistration.ValidationFinding                                           100%
  ExperimentFramework.ServiceRegistration.Validators.LifetimeSafetyValidator                          100%
  ExperimentFramework.Telemetry.NoopExperimentTelemetry                                               100%
  ExperimentFramework.Telemetry.OpenTelemetryExperimentTelemetry                                      100%
  ExperimentFramework.Validation.TrialConflictDetector                                               98.7%
  ExperimentFramework.Validation.TrialConflictException                                               100%
  ExperimentFramework.Variants.VariantFeatureManagerAdapter                                           100%

ExperimentFramework.Admin                                                                            33.4%
  ExperimentFramework.Admin.ExperimentAdminEndpoints                                                  100%
  ExperimentFramework.Admin.GovernanceAdminEndpoints                                                    0%

ExperimentFramework.Audit                                                                               0%
  ExperimentFramework.Audit.CompositeAuditSink                                                          0%
  ExperimentFramework.Audit.LoggingAuditSink                                                            0%
  ExperimentFramework.Audit.ServiceCollectionExtensions                                                 0%

ExperimentFramework.AutoStop                                                                          100%
  ExperimentFramework.AutoStop.AutoStopOptions                                                        100%
  ExperimentFramework.AutoStop.Rules.MinimumSampleSizeRule                                            100%
  ExperimentFramework.AutoStop.Rules.StatisticalSignificanceRule                                      100%
  ExperimentFramework.AutoStop.ServiceCollectionExtensions                                            100%
  ExperimentFramework.AutoStop.VariantData                                                            100%

ExperimentFramework.Bandit                                                                           98.8%
  ExperimentFramework.Bandit.Algorithms.EpsilonGreedy                                                 100%
  ExperimentFramework.Bandit.Algorithms.ThompsonSampling                                             97.3%
  ExperimentFramework.Bandit.Algorithms.UpperConfidenceBound                                          100%
  ExperimentFramework.Bandit.ArmStatistics                                                            100%
  ExperimentFramework.Bandit.ServiceCollectionExtensions                                              100%

ExperimentFramework.Configuration                                                                    68.6%
  ExperimentFramework.Configuration.Building.ConfigurationExperimentBuilder                          85.3%
  ExperimentFramework.Configuration.Building.TypeResolver                                              69%
  ExperimentFramework.Configuration.ConfigurationFileWatcher                                         90.1%
  ExperimentFramework.Configuration.Exceptions.ConfigurationLoadException                             100%
  ExperimentFramework.Configuration.Exceptions.ExperimentConfigurationException                       100%
  ExperimentFramework.Configuration.Exceptions.TypeResolutionException                                100%
  ExperimentFramework.Configuration.Extensions.ConfigurationExtensionRegistry                        79.1%
  ExperimentFramework.Configuration.Extensions.ConfigurationExtensionServiceCollectionExtensions     92.8%
  ExperimentFramework.Configuration.Extensions.Handlers.ConfigurationKeySelectionModeHandler          100%
  ExperimentFramework.Configuration.Extensions.Handlers.CustomDecoratorHandler                       93.7%
  ExperimentFramework.Configuration.Extensions.Handlers.CustomSelectionModeHandler                   66.6%
  ExperimentFramework.Configuration.Extensions.Handlers.FeatureFlagSelectionModeHandler               100%
  ExperimentFramework.Configuration.Extensions.Handlers.GovernanceConfigurationHandler                 44%
  ExperimentFramework.Configuration.Extensions.Handlers.InMemoryBackplaneConfigurationHandler        11.1%
  ExperimentFramework.Configuration.Extensions.Handlers.LoggingBackplaneConfigurationHandler         11.1%
  ExperimentFramework.Configuration.Extensions.Handlers.LoggingDecoratorHandler                      91.6%
  ExperimentFramework.Configuration.Extensions.Handlers.OpenTelemetryBackplaneConfigurationHandler   11.1%
  ExperimentFramework.Configuration.Extensions.Handlers.TimeoutDecoratorHandler                       100%
  ExperimentFramework.Configuration.Loading.ConfigurationFileDiscovery                                100%
  ExperimentFramework.Configuration.Loading.ExperimentConfigurationLoader                            84.6%
  ExperimentFramework.Configuration.Models.CircuitBreakerDecoratorOptions                             100%
  ExperimentFramework.Configuration.Models.OutcomeCollectionDecoratorOptions                          100%
  ExperimentFramework.Configuration.Models.TimeoutDecoratorOptions                                    100%
  ExperimentFramework.Configuration.Schema.Generated.ConfigurationSchema                                0%
  ExperimentFramework.Configuration.Schema.Generated.SchemaRegistry                                     0%
  ExperimentFramework.Configuration.Schema.SchemaExporter                                               0%
  ExperimentFramework.Configuration.Schema.SchemaHasher                                               100%
  ExperimentFramework.Configuration.Schema.SchemaVersionTracker                                      94.5%
  ExperimentFramework.Configuration.ServiceCollectionExtensions                                      48.2%
  ExperimentFramework.Configuration.Validation.ConfigurationValidationError                           100%
  ExperimentFramework.Configuration.Validation.ConfigurationValidationResult                          100%
  ExperimentFramework.Configuration.Validation.ConfigurationValidator                                69.5%

ExperimentFramework.Data                                                                             98.1%
  ExperimentFramework.Data.Configuration.OutcomeCollectionDecoratorHandler                            100%
  ExperimentFramework.Data.Decorators.OutcomeCollectionDecoratorFactory                               100%
  ExperimentFramework.Data.ExperimentBuilderExtensions                                                100%
  ExperimentFramework.Data.Models.ExperimentOutcome                                                   100%
  ExperimentFramework.Data.Models.OutcomeAggregation                                                  100%
  ExperimentFramework.Data.Models.OutcomeQuery                                                        100%
  ExperimentFramework.Data.Recording.OutcomeRecorder                                                  100%
  ExperimentFramework.Data.Recording.OutcomeRecorderOptions                                           100%
  ExperimentFramework.Data.ServiceCollectionExtensions                                                100%
  ExperimentFramework.Data.Storage.InMemoryOutcomeStore                                              93.6%
  ExperimentFramework.Data.Storage.NoopOutcomeStore                                                   100%

ExperimentFramework.Distributed                                                                      94.6%
  ExperimentFramework.Distributed.InMemoryDistributedLockProvider                                    92.5%
  ExperimentFramework.Distributed.InMemoryDistributedState                                           95.4%
  ExperimentFramework.Distributed.ServiceCollectionExtensions                                         100%

ExperimentFramework.Distributed.Redis                                                                 100%
  ExperimentFramework.Distributed.Redis.RedisDistributedLockProvider                                  100%
  ExperimentFramework.Distributed.Redis.RedisDistributedState                                         100%
  ExperimentFramework.Distributed.Redis.ServiceCollectionExtensions                                   100%

ExperimentFramework.FeatureManagement                                                                  80%
  ExperimentFramework.FeatureManagement.ExperimentBuilderExtensions                                    50%
  ExperimentFramework.FeatureManagement.ServiceCollectionExtensions                                   100%
  ExperimentFramework.FeatureManagement.VariantFeatureFlagProvider                                    100%

ExperimentFramework.Governance                                                                       33.4%
  ExperimentFramework.Governance.Approval.AutomaticApprovalGate                                         0%
  ExperimentFramework.Governance.Approval.ManualApprovalGate                                            0%
  ExperimentFramework.Governance.Approval.RoleBasedApprovalGate                                        23%
  ExperimentFramework.Governance.ApprovalManager                                                     11.1%
  ExperimentFramework.Governance.ApprovalResult                                                         0%
  ExperimentFramework.Governance.GovernanceBuilder                                                   65.9%
  ExperimentFramework.Governance.GovernanceConfiguration                                                0%
  ExperimentFramework.Governance.LifecycleManager                                                    79.1%
  ExperimentFramework.Governance.Policy.ConflictPreventionPolicy                                        0%
  ExperimentFramework.Governance.Policy.ErrorRatePolicy                                              11.7%
  ExperimentFramework.Governance.Policy.PolicyEvaluator                                              30.7%
  ExperimentFramework.Governance.Policy.TimeWindowPolicy                                                0%
  ExperimentFramework.Governance.Policy.TrafficLimitPolicy                                           10.8%
  ExperimentFramework.Governance.ServiceCollectionExtensions                                          100%
  ExperimentFramework.Governance.Versioning.VersionManager                                              0%

ExperimentFramework.Metrics.Exporters                                                                99.3%
  ExperimentFramework.Metrics.Exporters.OpenTelemetryExperimentMetrics                                100%
  ExperimentFramework.Metrics.Exporters.PrometheusExperimentMetrics                                  99.2%

ExperimentFramework.OpenFeature                                                                        80%
  ExperimentFramework.OpenFeature.ExperimentBuilderExtensions                                          50%
  ExperimentFramework.OpenFeature.OpenFeatureProvider                                                 100%
  ExperimentFramework.OpenFeature.ServiceCollectionExtensions                                         100%

ExperimentFramework.Plugins                                                                            87%
  ExperimentFramework.Plugins.Abstractions.PluginEventArgs                                            100%
  ExperimentFramework.Plugins.Abstractions.PluginLoadFailedEventArgs                                  100%
  ExperimentFramework.Plugins.Configuration.PluginConfigurationValidator                              100%
  ExperimentFramework.Plugins.Configuration.PluginDiscoveryService                                    100%
  ExperimentFramework.Plugins.Configuration.PluginsConfig                                             100%
  ExperimentFramework.Plugins.HotReload.PluginReloadEventArgs                                         100%
  ExperimentFramework.Plugins.HotReload.PluginReloadFailedEventArgs                                   100%
  ExperimentFramework.Plugins.HotReload.PluginReloadService                                           100%
  ExperimentFramework.Plugins.HotReload.PluginWatcher                                                87.8%
  ExperimentFramework.Plugins.Integration.PluginBuilderExtensions                                     100%
  ExperimentFramework.Plugins.Integration.PluginManager                                              83.3%
  ExperimentFramework.Plugins.Integration.PluginTypeResolver                                          100%
  ExperimentFramework.Plugins.Loading.PluginContext                                                  55.4%
  ExperimentFramework.Plugins.Loading.PluginLoadContext                                                72%
  ExperimentFramework.Plugins.Loading.PluginLoader                                                   91.9%
  ExperimentFramework.Plugins.Loading.SharedTypeRegistry                                             89.1%
  ExperimentFramework.Plugins.Manifest.ManifestLoader                                                  92%
  ExperimentFramework.Plugins.Manifest.ManifestValidationResult                                        50%
  ExperimentFramework.Plugins.Manifest.ManifestValidator                                             97.7%
  ExperimentFramework.Plugins.Manifest.PluginManifest                                                 100%
  ExperimentFramework.Plugins.Manifest.PluginManifestAttribute                                        100%
  ExperimentFramework.Plugins.Manifest.PluginManifestJson                                             100%
  ExperimentFramework.Plugins.Security.PluginSecurityValidator                                         73%
  ExperimentFramework.Plugins.ServiceCollectionDecoratorExtensions                                   70.5%
  ExperimentFramework.Plugins.ServiceCollectionExtensions                                             100%

ExperimentFramework.Resilience                                                                         99%
  ExperimentFramework.Resilience.CircuitBreakerDecoratorFactory                                       100%
  ExperimentFramework.Resilience.CircuitBreakerOpenException                                          100%
  ExperimentFramework.Resilience.CircuitBreakerOptions                                                100%
  ExperimentFramework.Resilience.Configuration.CircuitBreakerDecoratorHandler                         100%
  ExperimentFramework.Resilience.ResilienceBuilderExtensions                                          100%
  ExperimentFramework.Resilience.ResilienceServiceCollectionExtensions                               85.7%

ExperimentFramework.Rollout                                                                          98.7%
  ExperimentFramework.Rollout.Configuration.RolloutSelectionModeHandler                               100%
  ExperimentFramework.Rollout.Configuration.StagedRolloutSelectionModeHandler                        97.9%
  ExperimentFramework.Rollout.ExperimentBuilderExtensions                                             100%
  ExperimentFramework.Rollout.RolloutAllocator                                                         95%
  ExperimentFramework.Rollout.RolloutProvider                                                         100%
  ExperimentFramework.Rollout.ServiceCollectionExtensions                                             100%
  ExperimentFramework.Rollout.StagedRolloutOptions                                                    100%
  ExperimentFramework.Rollout.StagedRolloutProvider                                                   100%

ExperimentFramework.Science                                                                          88.8%
  ExperimentFramework.Science.Analysis.ExperimentAnalyzer                                            88.3%
  ExperimentFramework.Science.Builders.EndpointBuilder                                                100%
  ExperimentFramework.Science.Builders.HypothesisBuilder                                              100%
  ExperimentFramework.Science.Builders.SuccessCriteriaBuilder                                         100%
  ExperimentFramework.Science.Corrections.BenjaminiHochbergCorrection                                 100%
  ExperimentFramework.Science.Corrections.BonferroniCorrection                                        100%
  ExperimentFramework.Science.Corrections.HolmBonferroniCorrection                                    100%
  ExperimentFramework.Science.EffectSize.CohensD                                                      100%
  ExperimentFramework.Science.EffectSize.EffectSizeExtensions                                         100%
  ExperimentFramework.Science.EffectSize.OddsRatio                                                    100%
  ExperimentFramework.Science.EffectSize.RelativeRisk                                                 100%
  ExperimentFramework.Science.Models.Hypothesis.Endpoint                                                0%
  ExperimentFramework.Science.Models.Hypothesis.HypothesisDefinition                                    0%
  ExperimentFramework.Science.Models.Results.StatisticalTestResult                                   66.6%
  ExperimentFramework.Science.Models.Snapshots.EnvironmentInfo                                          0%
  ExperimentFramework.Science.Power.PowerAnalyzer                                                    78.7%
  ExperimentFramework.Science.Reporting.JsonReporter                                                 88.8%
  ExperimentFramework.Science.Reporting.MarkdownReporter                                             72.8%
  ExperimentFramework.Science.ServiceCollectionExtensions                                             100%
  ExperimentFramework.Science.Snapshots.InMemorySnapshotStore                                        90.1%
  ExperimentFramework.Science.Statistics.ChiSquareTest                                               79.1%
  ExperimentFramework.Science.Statistics.MannWhitneyUTest                                            97.3%
  ExperimentFramework.Science.Statistics.OneWayAnova                                                 98.4%
  ExperimentFramework.Science.Statistics.PairedTTest                                                 81.5%
  ExperimentFramework.Science.Statistics.TwoSampleTTest                                              88.4%

ExperimentFramework.StickyRouting                                                                    92.5%
  ExperimentFramework.StickyRouting.ExperimentBuilderExtensions                                       100%
  ExperimentFramework.StickyRouting.ServiceCollectionExtensions                                       100%
  ExperimentFramework.StickyRouting.StickyRoutingProvider                                            86.6%
  ExperimentFramework.StickyRouting.StickyTrialRouter                                                 100%

ExperimentFramework.Targeting                                                                         100%
  ExperimentFramework.Targeting.Configuration.TargetingSelectionModeHandler                           100%
  ExperimentFramework.Targeting.InMemoryTargetingConfiguration                                        100%
  ExperimentFramework.Targeting.ServiceCollectionExtensions                                           100%
  ExperimentFramework.Targeting.SimpleTargetingContext                                                100%
  ExperimentFramework.Targeting.TargetingProvider                                                     100%
  ExperimentFramework.Targeting.TargetingRules                                                        100%

@JerrettDavis JerrettDavis merged commit e4a9a03 into main Jan 15, 2026
6 checks passed
@JerrettDavis JerrettDavis deleted the copilot/add-experiment-framework-diagnostics branch January 15, 2026 19:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: tests dependencies documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Add ExperimentFramework.Diagnostics package with first-class event sinks and in-memory/logging/OTel bridges

3 participants