Conversation
lillo42
requested review from
DevJonny,
holytshirt,
iancooper and
preardon
as code owners
![codescene-delta-analysis[bot]](https://avatars.githubusercontent.com/in/53921?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Pull Request Review: Add MessageGateway Test GeneratorSummaryThis PR introduces a test generator for messaging gateway implementations, extending the existing test generation infrastructure to reduce code duplication across different messaging providers (RabbitMQ, AWS SNS/SQS, Azure Service Bus, etc.). The implementation follows the established pattern from the Outbox generator and includes comprehensive documentation via ADR 0037. Positive Aspects ✅Architecture & Design
Code Quality
Issues & Concerns 🔴1. Critical: Null Reference Issues in DefaultMessageFactory.csLocation: The public Message Create(MessageConfiguration? configuration = null)
{
var messageHeader = new MessageHeader(
messageId: configuration.MessageId, // ⚠️ NullReferenceException if configuration is null
topic: configuration.Topic,
// ... all other properties accessed without null checkFix: Either:
Severity: HIGH - This will cause runtime exceptions 2. Code Style Violation: Inconsistent SpacingLocation: The public class MessageConfiguration
{
public Dictionary<string, object> Bag { get; set; } = new() { ... };
public Baggage Baggage { get; set; } = new(); // ⚠️ 2 spaces instead of 4
public ContentType ContentType { get; set; } = ... // ⚠️ InconsistentFix: Use consistent 4-space indentation for all properties Severity: MEDIUM - Code style violation 3. Typo in Configuration Property NameLocation: Property name has a typo: public Dictionary<string, MessagingGatewayConfiguration>? MessagingGatewaies { get; set; }Impact: This affects the public API of the configuration and should be corrected before release Severity: MEDIUM - Breaking change required 4. Hard-coded Delay ValuesLocation: The generated test has hard-coded delays:
While the configuration has Severity: LOW - Test maintainability issue 5. Missing XML Documentation CommentLocation: The Severity: LOW - Documentation completeness 6. Commented-out Code in GeneratorLocation: // await GenerateAsync(
// configuration,
// Path.Combine("MessagingGateway", prefix, "Generated", "Reactor"),
// Path.Combine("MessagingGateway", "Reactor"),
// configuration.MessagingGateway,
// filename => SkipTest(configuration.MessagingGateway, filename)
// );Questions:
Severity: LOW - Code cleanliness 7. Empty Skip Test ImplementationLocation: private static bool SkipTest(MessagingGatewayConfiguration configuration, string fileName)
{
return false;
}This always returns
Severity: LOW - Potential missing functionality Test Coverage Concerns
|
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Pull Request Review: Add MessageGateway Test GeneratorSummaryThis PR successfully extends the test generator tool to support messaging gateway tests, following the established pattern from the Outbox generator. The implementation is well-architected, consistent with the existing codebase, and includes comprehensive documentation via ADR 0037. ✅ StrengthsArchitecture & Design
Code Quality
Testing Infrastructure
🔍 Issues & Concerns1. Copy-paste error in log message (Minor)Location: logger.LogInformation("Generating outbox test for {OutboxName}", key);Should be: logger.LogInformation("Generating messaging gateway test for {GatewayName}", key);2. Commented-out code (Minor)Location: Reactor pattern generation is commented out for single gateway configuration but enabled for multiple gateways (lines 87-93). This inconsistency suggests:
Recommendation: Remove commented code or add a TODO comment explaining why Reactor is disabled. 3. Typo in property name (Medium)Location: public Dictionary<string, MessagingGatewayConfiguration>? MessagingGateways { get; set; }Property is named 4. Potential null reference issues (Low)Location: Generated test template line 698 _subscription = _messageGatewayProvider.CreateSubscription(_publication.Topic!, ...)Using null-forgiving operator ( 5. Hardcoded delay values (Medium - Performance)Location: Generated test at lines 717 and 732 await Task.Delay(5000); // Hardcoded 5-second delayThe test hardcodes a 5-second delay even though the configuration specifies Recommendation: Use the configuration value from 6. Empty SkipTest implementation (Low)Location: private static bool SkipTest(MessagingGatewayConfiguration configuration, string fileName)
{
return false;
}This stub exists in
7. Missing validation (Low)Location: The configuration properties
💡 Suggestions for Improvement1. Test CoverageThe PR only includes one generated test:
2. Partition Key HandlingLocation: Line 707-710 of generated test _messageFactory.Create(new MessageConfiguration{Topic = _publication.Topic, PartitionKey = null}),The partition key is explicitly set to null. This seems intentional but isn't documented. Consider:
3. Resource Cleanup OrderingLocation: Channel is disposed synchronously ( 🔒 Security ConsiderationsNo security concerns identified. The generated tests use localhost RabbitMQ connections with guest credentials, which is appropriate for test environments. ⚡ Performance ConsiderationsTest execution time: The hardcoded 5-second delays mean each 4-message test takes ~20+ seconds. For large test suites, this could be significant. Consider:
📋 Documentation & MaintainabilityExcellent documentation overall. The ADR is thorough and includes:
Minor improvement: Add a section in the ADR about how to add new test scenarios in the future. 🎯 RecommendationsMust Fix (Before Merge)
Should Fix (Before or Shortly After Merge)
Nice to Have (Future Work)
✅ Final AssessmentRecommendation: APPROVE with minor changes This is a high-quality PR that:
The issues identified are mostly minor and can be addressed in follow-up commits or as part of this PR. The core architecture is sound and ready for use. Estimated risk level: Low Great work @lillo42! This is a valuable addition to the test infrastructure. 🎉 |
|
I just want to say how excited I am for this. |
iancooper
left a comment
iancooper
left a comment
There was a problem hiding this comment.
I can think of a few other points, but let's start with these.
| - Generates tests from Liquid templates | ||
| - Follows the same pattern as `OutboxGenerator` for consistency | ||
|
|
||
| ### 3. Updated Message Factories and Assertions |
There was a problem hiding this comment.
I note that I have had more success when running the Dispatcher using a handler that takes the FakeTimeProvider to advance the time, rather than using Task Delay, when timing is being used to trigger a behaviour. It's possible a few tests could benefit from this
|
|
||
| ### Negative | ||
|
|
||
| - **Learning Curve**: Developers need to understand the generator and template system |
There was a problem hiding this comment.
Where to start when you want to add a new feature to our MessagingGateways as well. Much harder to iterate on in-memory and then roll out. We need to document the recommended loop
| _messageAssertion = new DefaultMessageAssertion(); | ||
| } | ||
|
|
||
| public Task InitializeAsync() |
There was a problem hiding this comment.
This seems a little bit unnecessary. What forces this?
There was a problem hiding this comment.
It's part of the IAsyncLifeTime
| if (subscription.MakeChannels == OnMissingChannel.Create) | ||
| { | ||
| // Ensuring that the queue exists before return the channel | ||
| await channel.ReceiveAsync(TimeSpan.FromMilliseconds(100), cancellationToken); |
There was a problem hiding this comment.
It seems problematic to have to use Receive to force the call to EnsureChannel. I wonder if we need an Init method for a channel that also calls EnsureChannel. (Receive should still call EnsureChannel, but we would have an explicit Init too)
| ); | ||
| } | ||
|
|
||
| public ChannelName GetOrCreateChannelName([CallerMemberName] string? testName = null) |
There was a problem hiding this comment.
Why do we need this, over a property set via the constructor?
There was a problem hiding this comment.
It'll set the test name at compilation time, it's an option I prefer via this attribute because I don't need to force those who implement it to have a constructor
tests/Paramore.Brighter.RMQ.Async.Tests/DefaultMessageFactory.cs
Outdated
Show resolved
Hide resolved
| /// <summary> | ||
| /// Defines a contract for asserting equality between two Message instances. | ||
| /// </summary> | ||
| public interface IAmAMessageAssertion |
There was a problem hiding this comment.
Why is this an interface? Not sure I understand the thinking
There was a problem hiding this comment.
We need to assert the received message against the provided message. In some message gateways like RabbitMQ, the MessageId will be different on message Requeue, so I want to use this interface to cover this cases
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Pull Request Review: Add MessageGateway Test GeneratorSummaryThis PR extends the ✅ Strengths
🔍 Issues & ConcernsCritical Issues1. Naming Inconsistency: MessageFactory vs MessageBuilder (Lines:
Impact: This creates confusion in documentation and may cause issues when developers read the ADR vs implement the configuration. Recommendation: // Update ADR to consistently use "MessageBuilder" terminology
// OR rename MessageBuilder to MessageFactory throughout the codebase2. Missing XML Documentation on Configuration Properties ( public bool HasSupportToPublishConfirmation { get; set; }
public bool HasSupportToDelayedMessages { get; set; }
public bool HasSupportToPartitionKey { get; set; }
public bool HasSupportToDeadLetterQueue { get; set; }
public bool HasSupportToValidateBrokerExistence { get; set; }Impact: Developers won't understand what these flags control without reading the generator code. Recommendation: Add XML documentation explaining each flag's purpose and default behavior. High Priority Issues3. Hardcoded Delay Value ( await Task.Delay(5000);This 5-second delay is hardcoded in generated tests, despite Impact: Tests will always wait 5 seconds regardless of configuration, making tests slower than necessary. Recommendation: Use the configured delay value from 4. Incomplete Dead Letter Queue Implementation ( public Task<Message> GetMessageFromDeadLetterQueueAsync(...)
{
throw new NotImplementedException();
}Impact: Tests requiring DLQ validation will fail at runtime with Recommendation: Either implement these methods or conditionally skip DLQ tests when the provider doesn't support them. 5. Missing Default Values in Configuration Impact: Null reference exceptions if configuration is incomplete. Recommendation: Add validation in Medium Priority Issues6. ADR Date Mismatch 7. Timestamp Comparison Precision Loss ( Xunit.Assert.Equal(expected.Header.TimeStamp.ToString("yyyy-MM-ddTHH:mm:ss"),
actual.Header.TimeStamp.ToString("yyyy-MM-ddTHH:mm:ss"));Impact: Loses millisecond precision which might be relevant for some scenarios. Recommendation: Consider whether millisecond precision matters for your use case. If not, add a comment explaining the precision loss is intentional. 8. Generator Log Message Inconsistency ( logger.LogInformation("Generating outbox test for {OutboxName}", key);Should say "messaging gateway test" not "outbox test". 🎯 RecommendationsDocumentation
Configuration
Testing
Code Quality
📊 Performance ConsiderationsThe 5-second delay in tests could make test suites slow. Consider:
🔒 Security ConsiderationsNo security concerns identified. The generator creates test code only and doesn't handle sensitive data. 📝 Test Coverage AssessmentGenerated tests cover the main scenarios well:
🎉 ConclusionThis is a well-designed feature that will provide significant value to the Brighter project. The architecture is solid, the code is clean, and the documentation is comprehensive. The issues identified are mostly minor and can be addressed with targeted fixes. Recommendation: Approve with minor changes The critical issues around naming consistency and missing XML documentation should be addressed before merge, but they don't block the overall architecture. The incomplete DLQ implementation is acceptable if documented as future work. Great work! This will make gateway implementation testing much more consistent and maintainable. Review generated by Claude Code 🤖 |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
|
test comment |
Code Review: Add MessageGateway Test Generator Part 1 of 2Thanks for this PR - the concept of a test generator for messaging gateway implementations is a great idea that will ensure consistent coverage across providers. Here is my detailed feedback. BugsBug 1: async void Dispose() in Reactor Test Templates The Reactor test templates (and generated files) have async void Dispose() but the method body is entirely synchronous. async void methods are a well-known C# anti-pattern because unhandled exceptions become unobserved and can crash the process. Fix: Remove the async keyword (keep public void Dispose()). Affected templates:
Bug 2: Class Name Mismatch The file is named MessagingGatewayGenerator.cs but the class inside is MessageGatewayGenerator (missing ing). Rename the class to MessagingGatewayGenerator. Bug 3: Copy-Paste Error in Log Message In MessagingGatewayGenerator.cs the foreach loop for multiple gateways contains a copy-paste from OutboxGenerator: logger.LogInformation says Generating outbox test for OutboxName but should say Generating messaging gateway test for GatewayName. |
Code Review: Add MessageGateway Test Generator Part 2 of 2Design IssuesIssue 4: Vacuous Assert.True(true) in Exception Tests The should throw exception tests use a pattern that always passes regardless of whether any exception is thrown. The catch block uses Assert.True(true) which provides zero assertion value. This appears in 6 templates/generated files (both Proactor and Reactor variants for assume_channel, validate_channel, no_broker_created). Use Assert.ThrowsAnyAsync or Record.ExceptionAsync instead, or omit the catch body. Issue 5: GetOrCreateRoutingKey and GetOrCreateChannelName Ignore the testName Parameter In RmqMessageGatewayProvider, both methods ignore testName entirely and always return fresh random UUIDs. The contract implied by GetOrCreate and the CallerMemberName attribute is that calling the method multiple times with the same test name returns the same value. A test that calls GetOrCreateRoutingKey() twice would get mismatched routing keys. Either implement a dictionary-based cache keyed on testName, or rename to CreateRoutingKey / CreateChannelName. Issue 6: GetMessageFromDeadLetterQueue is Unimplemented Both GetMessageFromDeadLetterQueueAsync and GetMessageFromDeadLetterQueue in RmqMessageGatewayProvider throw NotImplementedException. Either implement them or set HasSupportToDeadLetterQueue = false in test-configuration.json to skip those tests. Issue 7: test-configuration.json Missing DestinationFolder and Category Without DestinationFolder, the generator falls back to Directory.GetCurrentDirectory() at runtime. Without Category, generated tests will not get a Trait Category RabbitMQ attribute, breaking CI filtering. The ADR example configuration (line 147) shows category RabbitMQ but it is absent from the actual config file. Issue 8: Generated Files Committed Without Documenting the Intent The PR commits 20+ auto-generated files alongside the templates. This creates a divergence risk. Please clarify in the ADR or a README in the Generated folder: are these files intentionally committed as a reference snapshot, or should they be gitignored and regenerated as part of the build? Minor Nits
OverallThis is a well-structured feature that follows the established OutboxGenerator design and the ADR is thorough. The main items to address before merging:
|
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Code Review — PR #3996: Add MessageGateway Test GeneratorThis is a well-motivated PR that extends the existing test-generator infrastructure to eliminate copy-paste duplication across gateway test suites. The overall architecture is sound and follows established patterns from the outbox generator. Below are issues ranging from genuine bugs to style nits. Bugs1. Copy-paste log message in logger.LogInformation("Generating outbox test for {OutboxName}", key);This was copied from 2. Mutation side-effect on messagingGatewayConfiguration.Prefix = $".{prefix}";This mutates the var effectivePrefix = string.IsNullOrEmpty(messagingGatewayConfiguration.Prefix) ? key : messagingGatewayConfiguration.Prefix;
// use effectivePrefix below, do not write back into the config3.
command.CommandTimeout = Convert.ToInt32(timeOut.Value.TotalSeconds); // async ✓But the sync command.CommandTimeout = Convert.ToInt32(timeOut.Value.Seconds); // sync ✗
Class / Type Naming4. Class name mismatch: file is public class MessageGatewayGenerator(ILogger<MessageGatewayGenerator> logger) // missing "ing"The summary doc-comment, the file name, and the ADR all call it Missing XML DocumentationThree properties in public string? CollectionName { get; set; } // line 63 — no doc
public bool HasSupportToValidateInfrastructure { get; set; } = true; // line 110 — no doc
public int ReceiveMessageTimeoutInMilliseconds { get; set; } = 300; // line 112 — no docAsync Correctness5. public async Task<IAmAMessageProducerAsync> CreateProducerAsync(
SnsPublication publication, CancellationToken cancellationToken = default)
{
var producer = new SnsMessageProducer(connection, publication);
return producer; // no await — unnecessary state machine
}This pattern appears in multiple provider files. Either remove 6. Blocking sync-over-async in public Message GetMessageFromDeadLetterQueue(SqsSubscription subscription)
{
return GetMessageFromDeadLetterQueueAsync(subscription).GetAwaiter().GetResult();
}This can deadlock on a synchronization-context thread. If the sync interface truly requires it, at least use Generated Test Patterns (Template Issues)These affect all generated tests and should be fixed in the Liquid templates. 7. // Assert
Assert.True(true);"No exception was thrown" is a legitimate test outcome, but 8. Fragile exception-testing pattern in the "no broker" test catch (Exception ex) when (ex is not Xunit.Sdk.XunitException)
{
Assert.True(true);
}This catches any exception (including await Assert.ThrowsAnyAsync<Exception>(() => _producer.SendAsync(message));Or, if a specific exception type is expected (e.g. Interface Design Nit9. RoutingKey GetOrCreateRoutingKey([CallerMemberName]string testName = null!);
RoutingKey GetOrCreateRoutingKey([CallerMemberName]string? testName = null);and handle nullability explicitly in implementations. ADR InconsistenciesThe ADR ( Minor Positive Notes
Summary of blocking items before merge:
Items 7 and 8 (template-level test patterns) should also be addressed before the generated tests are considered reliable. |
There was a problem hiding this comment.
Gates Passed
See analysis details in CodeScene
Quality Gate Profile: Clean Code Collective
Install CodeScene MCP: safeguard and uplift AI-generated code. Catch issues early with our IDE extension and CLI tool.
2 participants
Overview
Extends the \Paramore.Brighter.Test.Generator` tool to automatically generate standardized tests for messaging gateway implementations (RabbitMQ, AWS SNS/SQS, Azure Service Bus, etc.), reducing code duplication and
ensuring consistent test coverage across all gateway implementations.
What's Changed
Configuration System
MessagingGatewayandMessagingGatewaiespropertiesGenerator Implementation
Message Factory & Assertions
IAmAMessageFactoryinterface andDefaultMessageFactoryimplementationIAmAMessageAssertioninterface andDefaultMessageAssertionimplementationCreatedMessagespropertyTemplates
MessagingGateway/Proactor/- Async test templatesMessagingGateway/Reactor/- Sync test templatesDocumentation
Benefits