chore: added experimental OTLP trace exporter support

[abhilash-sivan]

Summary

Introduces the Instana-to-OTLP exporter.

• Adds support for exporting traces via OTLP (4318/v1/traces).
• Supports OpenTelemetry Semantic Conventions (SemConv) v1.23.

Out of Scope

• Metrics export
• Logs export

TODO

Core Implementation

• Handle agent port usage (data port vs. connection port) (chore: make agent data port configurable independently from the agent connection port #2559)
• Support multiple span data keys (e.g., span.data.peer, span.data.mongo)
• Finalize overall design
• Implement SemConv lookup handling
• Implement span name mapping
• Complete metadata attribute mapping:
• Direct mappings
• Computed mappings
• Research span status mapping
• Research span name mapping (@abhilash-sivan)
• Implement OpenTelemetry instrumentation-specific handling
• Implement log-specific handling
• Implement resource mapping
• Implement required metrics mappings
• Implement agent connection logic
• Error handling @abhilash-sivan
• Resource-service name mapping
• Dual transformation flow in span data @abhilash-sivan
• Implement configuration option for enabling OTLP export (INSTANA_OTLP_ENABLED?) - NOT PART OF THIS
• remove example apps
• agent data port issue
• stack trace capture
• exception
• Fix type definitions
• error handling with 400 status
• Pipeline green

Span Data Mapping @abhilash-sivan

• HTTP
• Kafka
• Databases
• Cloud services

Testing

Unit Tests

• Single-span scenarios
• Multi-span scenarios
• unit test coverage
• integration tests
• SpanBuffer test

End-to-End Testing (@abhilash-sivan)

• SemConv v1.23 validation with the current design
• SemConv v1.41 compatibility validation
• OpenTelemetry wrapper instrumentation testing
• Metadata validation with SemConv v1.23 and span data validation with v1.43
• without metrics - corelation issue

Performance Testing

• Performance testing

Issue Tracking

• UI Issues: https://ibm.box.com/s/0v5wli0pgvrlg8jqpkhi59mb52o28u75
• Results: https://ibm.box.com/s/0v5wli0pgvrlg8jqpkhi59mb52o28u75

Validation Checklist

Before merging, verify the following:

• Traces are successfully exported through the OTLP endpoint (4318/v1/traces).
• Metrics continue to be exported through the existing endpoint.
• The UI correctly identifies and displays applications.

Future Enhancements/TODO

• Support the agent announcement cycle disablement use case.
• Evaluate exporting logs to the OTLP /v1/logs endpoint.
• Implement metrics mapping.
• Refactor the current dual transformation flow in span data (internal mapping and OTLP mapping).

References

• https://opentelemetry.io/blog/2023/http-conventions-declared-stable/
• https://github.com/open-telemetry/opentelemetry-proto/blob/main/opentelemetry/proto/trace/v1/trace.proto

[kirrg001]

test: send traces via otlp port and metrics via standard port - Idenfies 2 diff apps 1. Otel and 2. Node.js

This needs to be communicated in the next WG. Traces & Metrics mapping is required.

[abhilash-sivan]

Not verified rabbitmq per se. It's just to show how we differentiate children under messaging or any other group.

[aryamohanan]

But for RabbitMQ, there’s nothing really common with BASE_MAPPINGS.messaging. I don’t think any of the others share much with it either, apart from Kafka.

[kirrg001]

Why do we need that here? If we have already a custom kafka mapping object?

[abhilash-sivan]

messaging.system is an additional attribute that we append to the span. For Kafka, the system name also happens to be kafka, so it may seem redundant at first glance.

However, additionalAttributes is intended for constant or computed attributes that do not directly map from a single field inside span.data.kafka.

messaging.system falls into that category since it is effectively a constant defined per transformer, rather than something extracted from the incoming span payload itself.

I have changed this to: 'messaging.system': this.systemName

which will fetch the appropriate value in this case

[kirrg001]

Do we need this app still?
Wondering if we can just have a separate npm script in the example-apps/collector app?

With config otlp being on.

[aryamohanan]

We can remove this after finishing the playground work, since the example app also includes Kafka and database tests that are only required for playground testing.

[kirrg001]

transmissionCycle has its own config reference. Why do we provide the extra config object here?

[aryamohanan]

I passed _config because the configuration is evaluated after the agent response is received. Using the config passed at activation time ensures we use the configuration that was resolved according to the precedence rules.

[aryamohanan]

I removed this logic because the separation between data port and agent connection port does not work well for the current OTLP implementation.

sendData() is used for multiple endpoint types. Some requests need to be sent to OTLP endpoints, while others still need to go to regular agent endpoints (eg: /tracemetrics). Switching the port based on OTLP state at a higher level can cause non-OTLP requests to be sent to the wrong port.

In addition, this code executes before the final configuration has been resolved. At this stage, we do not yet know the effective value of otlp.enabled, so making port-selection decisions here can be incorrect.

For these reasons, I removed the logic and kept endpoint/port selection closer to the actual export path, where both the resolved configuration and the target endpoint are known.