Java is a special case in OpenTelemetry. The instrumentation is particularly good, but its configuration is unique.
How do I customize my pipeline?
Today, I want to add a field to every span. (Not the same value for each one, or I’d define `OTEL_RESOURCE_ATTRIBUTES`.) Like, I want to grab the count of running threads and throw that on every span. This calls for a SpanProcessor.
Writing a SpanProcessor is easy enough; there’s an interface to implement. Make an onStart method that adds attributes to the passed-in span.
Getting that SpanProcessor to run is tricky. In languages like JavaScript, I can configure processors and exporters as part of initializing the OpenTelemetry SDK.
It’s possible to configure the OpenTelemetry Java SDK in code, but useless, because then you don’t get any automatic instrumentation! That part comes only from the OpenTelemetry Java Agent.
In Java, we build our code into a jar file, and then run that with:java ourOwnCode.jar
That starts up the JVM, loads our code, and calls a main method somewhere.
OpenTelemetry has its own jar file, the Java Agent, which we glom onto our program at runtime with an additional JVM argument, like this:
java -javaagent opentelemetry-javaagent.jar ourOwnCode.jar
If I want to add a custom SpanProcessor, or any other modifications to telemetry processing, I have to put that code in yet a third jar. Then the command looks like this:
java -javaagent opentelemetry-javaagent.jar -Dotel.javaagent.extensions=fancySpanProcessor.jar ourOwnCode.jar
The new JVM argument has three parts: -D says “Here comes a property for you to hold on to, so that the code running in this process can read it.” The name of that property is otel.javaagent.extensions. On startup, the OpenTelemetry Java agent reads that property, finds a path to a `jar`, dynamically loads that jar, and then does some magic with the classes it finds there.
In particular, it looks for implementations of AutoConfiguratorCustomizerProvider. So I need one of those in the jar with my SpanProcessor. It can get access to the tracerProvider, which accepts my additional SpanProcessor. There’s an official example of this here.
My implementation for the custom SpanProcessor extension is in this repo. You can also see a miniature Java project that uses it.
