RFC: Tracing Sampling Strategy Rules

[david-martin]

This RFC is an attempt to capture the thoughts and considerations around the bullet item 'Look into adding "well-known" fields into our span to "trace" them back to the user's domain: RLP, AP, …' from #48

The body of work described in the doc is post v1 milestone. Priority & scheduling TBD.

TODO:

• Get answers to unresolved questions & update the doc accordingly
• Investigate 'prior art'

[Boomatang]

As this would be configured inside the kuadrant CR, it would be worth looking at #112 as it is trying to define a standardized API design for new features added to the kuadrant CR

[alexsnaps]

Wrt to the prior art, you might want to look into head- vs. tail-based sampling in tracing. But I'm somewhat confused about two main use cases:

• Error (somewhere late in the game something "fails", which that would need definition already - an exception/panic/err would be already complicated bu what about e.g. "too slow" request) and as a user I'd certainly would want to have that trace available
• Enabling tracing on a "late in the game known attribute", e.g. trace for user bob, how would limitador know that pre-auth ratelimiting phase would need to send traces for ... bob that we'd only know about in the auth phase?

I think this tries to resolve tail-based sampling... If so, do we want to do that? Now that being said, there is still much more we can do to be a better citizen in that space.