Ensure the handler invocation infrastructure does not pollute the stack trace

[danielmarbach]

I realized .NET has had for a while the StackTraceHiddenAttribute which makes in the majority of the cases things disappear from the stack trace and the stack trace when it is turned into a string. Since we replaced the exception tree part with a bit of class infrastructure, we are essentially adding more "weight" to the stack trace which has implications on transports that have limited message sizes because it might make it more likely to go over the message size limits.

This pull request improves the clarity of stack traces for error messages in NServiceBus by ensuring internal framework classes are hidden from stack traces, making it easier for users to diagnose issues. The changes include new attributes on key handler invoker and factory methods, and an acceptance test to verify the improvement.

Stack trace cleanliness improvements:

• Added [DebuggerNonUserCode], [DebuggerStepThrough], and [StackTraceHidden] attributes to the Invoke method in MessageHandlerInvoker to hide it from stack traces.
• Refactored handler invocation in TimeoutHandlerFactory and MessageHandlerFactory to use dedicated static methods (InvokeTimeout and InvokeHandler) with the same stack trace hiding attributes, ensuring these methods do not appear in stack traces. [1] [2]

Test coverage:

• Added a new acceptance test When_message_is_moved_to_error_queue.cs to verify that stack traces for moved error messages do not contain internal NServiceBus classes like MessageHandlerInvoker and MessageHandlerFactory.

Documentation and maintainability:

• Added comments to warn against renaming MessageHandlerInvoker, TimeoutHandlerFactory, and MessageHandlerFactory since they are referenced in acceptance tests for stack trace verification. [1] [2] [3] [4]

Before:

NServiceBus.AcceptanceTesting.SimulatedException: Exception of type 'NServiceBus.AcceptanceTesting.SimulatedException' was thrown.
   at NServiceBus.AcceptanceTests.Recoverability.When_custom_policy_discards_failed_message.FailingMessageHandler.Handle(FailingMessage message, IMessageHandlerContext context) in /Users/danielmarbach/Projects/NServiceBus/src/NServiceBus.AcceptanceTests/Recoverability/When_custom_policy_discards_message.cs:line 51
   at NServiceBus.Unicast.MessageHandlerRegistry.MessageHandlerFactory`2.<>c.<InvokeHandler>b__4_0(IHandleMessages`1 handler, TMessage message, IMessageHandlerContext handlerContext) in /Users/danielmarbach/Projects/NServiceBus/src/NServiceBus.Core/Unicast/MessageHandlerRegistry.cs:line 247
   at NServiceBus.MessageHandlerInvoker`2.Invoke(Object message, IMessageHandlerContext handlerContext) in /Users/danielmarbach/Projects/NServiceBus/src/NServiceBus.Core/Pipeline/Incoming/MessageHandlerInvoker.cs:line 45
   at NServiceBus.InvokeHandlerTerminator.Terminate(IInvokeHandlerContext context) in /Users/danielmarbach/Projects/NServiceBus/src/NServiceBus.Core/Pipeline/Incoming/InvokeHandlerTerminator.cs:line 31
   at NServiceBus.LoadHandlersConnector.Invoke(IIncomingLogicalMessageContext context, Func`2 stage) in /Users/danielmarbach/Projects/NServiceBus/src/NServiceBus.Core/Pipeline/Incoming/LoadHandlersConnector.cs:line 59

after:

NServiceBus.AcceptanceTesting.SimulatedException: Exception of type 'NServiceBus.AcceptanceTesting.SimulatedException' was thrown.
   at NServiceBus.AcceptanceTests.Recoverability.When_custom_policy_discards_failed_message.FailingMessageHandler.Handle(FailingMessage message, IMessageHandlerContext context) in /Users/danielmarbach/Projects/NServiceBus/src/NServiceBus.AcceptanceTests/Recoverability/When_custom_policy_discards_message.cs:line 51
   at NServiceBus.InvokeHandlerTerminator.Terminate(IInvokeHandlerContext context) in /Users/danielmarbach/Projects/NServiceBus/src/NServiceBus.Core/Pipeline/Incoming/InvokeHandlerTerminator.cs:line 31
   at NServiceBus.LoadHandlersConnector.Invoke(IIncomingLogicalMessageContext context, Func`2 stage) in /Users/danielmarbach/Projects/NServiceBus/src/NServiceBus.Core/Pipeline/Incoming/LoadHandlersConnector.cs:line 59

[danielmarbach]

Wondering if this would be a great improvement to apply to internal behaviors too because it could significantly reduce the stack trace size

[andreasohlund]

Wondering if this would be a great improvement to apply to internal behaviors too because it could significantly reduce the stack trace size

👍