Add ExperimentFramework.Diagnostics package with event sinks for standardized observability#59
Conversation
Co-authored-by: JerrettDavis <2610199+JerrettDavis@users.noreply.github.com>
Co-authored-by: JerrettDavis <2610199+JerrettDavis@users.noreply.github.com>
Co-authored-by: JerrettDavis <2610199+JerrettDavis@users.noreply.github.com>
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.OpenSSF Scorecard
Scanned Files
|
There was a problem hiding this comment.
Pull request overview
This PR adds the ExperimentFramework.Diagnostics package to provide standardized event capture infrastructure for experiment lifecycle events. It enables consistent logging, test assertions, and telemetry without requiring bespoke implementations per application.
Changes:
- Introduces core event model with 7 event kinds and an
IExperimentEventSinkinterface - Provides three built-in sinks: InMemory (for testing), Logger (structured logging), and OpenTelemetry (metrics/activities)
- Adds DI extension methods for easy registration and composition of multiple sinks
- Includes comprehensive documentation and 40+ unit tests
Reviewed changes
Copilot reviewed 18 out of 18 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
src/ExperimentFramework.Diagnostics/ExperimentEvent.cs |
Core event model with discriminated union pattern for 7 event types |
src/ExperimentFramework.Diagnostics/IExperimentEventSink.cs |
Interface definition with pass-by-reference efficiency |
src/ExperimentFramework.Diagnostics/InMemoryExperimentEventSink.cs |
Thread-safe in-memory sink with bounded/unbounded storage |
src/ExperimentFramework.Diagnostics/LoggerExperimentEventSink.cs |
Structured logging sink with event IDs 1001-1007 |
src/ExperimentFramework.Diagnostics/OpenTelemetryExperimentEventSink.cs |
OpenTelemetry metrics and activities using BCL types |
src/ExperimentFramework.Diagnostics/CompositeExperimentEventSink.cs |
Composite pattern with deterministic ordering and exception isolation |
src/ExperimentFramework.Diagnostics/ExperimentEventPublisher.cs |
Bridge component with factory methods for event creation |
src/ExperimentFramework.Diagnostics/ExperimentDiagnosticsExtensions.cs |
DI extension methods for sink registration |
src/ExperimentFramework.Diagnostics/ExperimentFramework.Diagnostics.csproj |
Multi-target package configuration (net8.0, net9.0, net10.0) |
tests/ExperimentFramework.Diagnostics.Tests/* |
Comprehensive test coverage with 40+ tests for all sinks and scenarios |
docs/user-guide/diagnostics.md |
Complete user guide with examples and best practices |
README.md |
Updated main README with diagnostics section and example |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
src/ExperimentFramework.Diagnostics/ExperimentFramework.Diagnostics.csproj
Outdated
Show resolved
Hide resolved
src/ExperimentFramework.Diagnostics/CompositeExperimentEventSink.cs
Outdated
Show resolved
Hide resolved
tests/ExperimentFramework.Diagnostics.Tests/ExperimentFramework.Diagnostics.Tests.csproj
Outdated
Show resolved
Hide resolved
tests/ExperimentFramework.Diagnostics.Tests/LoggerExperimentEventSinkTests.cs
Outdated
Show resolved
Hide resolved
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #59 +/- ##
=======================================
Coverage ? 83.23%
=======================================
Files ? 184
Lines ? 6860
Branches ? 962
=======================================
Hits ? 5710
Misses ? 1150
Partials ? 0
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
@copilot apply changes based on the comments in this thread |
|
@copilot Retry |
…ling, standardize xunit version, add type check Co-authored-by: JerrettDavis <2610199+JerrettDavis@users.noreply.github.com>
Applied all code review feedback in commit b6b9354:
All 40 tests still passing. |
…entSinkTests.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Code Coverage |
Description
Adds
ExperimentFramework.Diagnosticspackage providing standardized event capture for experiment lifecycle events. Enables consistent logging, test assertions, and telemetry without bespoke per-app implementations.Type of Change
Related Issues
Changes Made
Core Event Model
ExperimentEventdiscriminated union with 7 event kinds:TrialStarted,TrialEnded,RouteSelected,FallbackOccurred,ExceptionThrown,MethodInvoked,MethodCompletedIExperimentEventSinkinterface withOnEvent(in ExperimentEvent e)for pass-by-reference efficiencyExperimentEventPublisherbridge component with factory methods for framework integrationBuilt-in Sinks
ConcurrentQueue, query API for test assertionsSystem.Diagnostics)DI Extensions
AddInMemoryExperimentEventSink(maxCapacity?),AddLoggerExperimentEventSink(categoryName?),AddOpenTelemetryExperimentEventSink()GetExperimentEventSinks()returns composite of all registered sinksDocumentation
docs/user-guide/diagnostics.mdwith examples, performance notes, best practicesCode Quality Improvements
CompositeExperimentEventSinkto filter critical exceptions (OutOfMemoryException, StackOverflowException) while providing debug output for other failuresExample Usage
Testing
Test Configuration
Checklist
Additional Context
Performance considerations:
inparameter) to avoid allocationsException handling strategy:
Acceptance criteria met:
Original prompt
ExperimentFramework.Diagnosticspackage with first-class event sinks and in-memory/logging/OTel bridges #58✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.