OpenTelemetry Visual Handbook · lab
You called startSpan. Who decides whether it leaves?
Not the tracer. A Tracer has no configuration at all — it
holds an instrumentation scope and nothing else. Everything that
decides the fate of a span is on the provider: the resource
stamped on it, the sampler consulted before it exists, the processors
it passes through, the limits it is trimmed to, and the moment
somebody calls flush.
There is a second thing this lab shows, and it is the reason it is the
first one in this handbook. Open the envelope on the right while the
simulation runs: ResourceSpans → ScopeSpans → Span is,
field for field, provider → tracer → span. The OTLP payload is
not a serialization of the SDK — it is the SDK’s object
graph, written down. That is why service.name sits one
layer above the spans, and why nobody can put it on a span.
Break it
Four perturbations, and none of them is scripted — each one is the same run with one control moved.
- Drag the sampling ratio down and tick RECORD_ONLY. The middle outlet lights up. Those spans are created, recorded and readable inside the process, and the exporter never sees one of them. Almost nobody knows that outlet exists.
- Set
maxQueueSizeto 4 and let it run. The queue refuses. The drop is a fact in the model, not a silence — it leaves by its own outlet and it is counted. - Kill the process, then kill it again with
ForceFlush. Same run, one parameter. Without the flush the queue dies with the process; with it, one control line descends to every registered processor and the batch leaves first. That is why spans vanish at the end of a short job. - Turn on
trace_based, then drop sampling to zero. Logs stop arriving. The only thing joining those two facts is a single dashed line crossing from one provider into another.
Predict before you look
Answer first, then run it. Predicting and being wrong is what makes the answer stick; watching and recognising is not the same thing.
A sampler returns RECORD_ONLY for a span. What reaches the exporter?
Commit to an answer first. You cannot change it afterwards.
Instrument it
Two decisions from the file that actually runs in
labs/providers/. The correct block is not written here — it
is lifted out of that file, so it cannot drift from the code that
compiles. The wrong ones are real: both of them ship to production
regularly.
The checkout service already installs an SDK at start-up. This method handles one request, and it needs a span.
Where does the Tracer come from?
Span span = tracer.spanBuilder("GET /checkout").startSpan();
try (var escopo = span.makeCurrent()) {
span.setAttribute("http.route", "/checkout");
} finally {
span.end();
}Pick one. You cannot change it afterwards — and that is the point.
labs/providers/app/src/main/java/checkout/Checkout.java · OpenTelemetry Java 1.65.0
The same service, at start-up. Everything it exports has to say which service it came from.
Where does service.name belong?
Resource recurso =
Resource.getDefault().merge(
);
SdkTracerProvider provider =
SdkTracerProvider.builder()
.setResource(recurso)
.setSampler(
Sampler.traceIdRatioBased(
Double.parseDouble(variavel("OTEL_TRACES_SAMPLER_ARG", "1.0"))))
.addSpanProcessor(
BatchSpanProcessor.builder(
OtlpGrpcSpanExporter.builder()
.setEndpoint(
variavel("OTEL_EXPORTER_OTLP_ENDPOINT", "http://localhost:4317"))
.build())
.setScheduleDelay(
Duration.ofMillis(Long.parseLong(variavel("OTEL_BSP_SCHEDULE_DELAY", "5000"))))
.build())
.build();Pick one. You cannot change it afterwards — and that is the point.
labs/providers/app/src/main/java/checkout/Checkout.java · OpenTelemetry Java 1.65.0
What people get wrong
Every row below is a belief worth naming, because the drawing above is arranged to take it apart. The right column is the spec, with the link.
| What people believe | What the spec says |
|---|---|
| service.name is an attribute of the span. | It belongs to the Resource, which belongs to the provider — and in OTLP it sits one layer above the spans. spec → |
| You configure the tracer. | A Tracer has no configuration. It holds an InstrumentationScope and nothing else; everything configurable lives on the provider. spec → |
| Sampling happens on export. | It happens at span creation, before any processor sees the span. spec → |
| A dropped span and a non-sampled span are the same thing. | RECORD_ONLY exists: the span is created, recorded and readable in-process — and the exporter never sees it. spec → |
| The provider propagates context. | Propagators are global and belong to no provider. Swapping a provider changes nothing about how context travels. spec → |
| If the SDK did not come up, something raises an error. | The API returns a no-op provider. No error, no warning — just a counter going up and nothing leaving. spec → |
| Metrics work like traces. | One is pulled and the other is pushed, and the default intervals differ by twelve times — 60000 ms against 5000 ms. spec → |
| The exporter retries when a send fails. | The spec says the default SDK must NOT implement retry — that logic depends on the protocol and belongs to the protocol exporter. spec → |
| Running out of memory always means losing data. | It depends on who is holding it. The span queue refuses and the span dies; the metric store collapses the excess into one overflow row and the sum is preserved. spec → |
What is deliberately not here
The channel out of the process is opaque, and it says so when you click it. HTTP/2 framing, length prefixes and stream ids are real, and they answer a different question than this lab asks — they are the gRPC over HTTP/2 annex and the Collector pipeline lab. What happens inside the Collector is not modelled here either: it is another program, it is drawn outside the process for that reason, and drawing it as an empty box would be worse than saying so.
Retry is missing on purpose too, and its absence is content: the spec says the default SDK must not implement retry, because the logic depends on the protocol. Modelling it here would be a lie about the SDK.
Check yourself
-
Two
TracerProviders in one process, with different resources. What fails? Nothing. You get tworesourceSpanswith differentservice.name, no error anywhere, and a backend that quietly stops correlating them. -
The queue is full and the metric store is at its cardinality limit.
Do they lose data the same way? No. The queue refuses and the
span is gone; the store collapses the excess into a single
otel.metric.overflowrow and the sum is preserved. Same problem — finite memory — two different lies. - Your SDK never came up. What is the first symptom? Silence. No exception, no warning, a no-op provider swallowing spans. It is the most common first-instrumentation failure, and the only way to teach it is to put a number on it.
Check your understanding
The assessment for this lab has not been authored yet.