OpenTelemetry Visual Handbook · lab
The waterfall you look at does not exist until you ask for it
A trace is not an object that lives somewhere while the request runs. Each span knows one edge — its parent — and carries it wherever it goes; each process exports on its own schedule; and the tree is the transitive closure of those edges, computed after the fact by whoever collected them.
Below, one request crosses four processes. Watch the calls move
forward and the spans leave downward, one per
service, each on its own wire. Click any item on a wire to
follow it: the panel shows what it is carrying right now, and every
hop it has made — with the field that changed at each one. The
traceparent is rewritten at every service, and that is
the whole mechanism, happening.
Then break it: drop the header at a boundary, or take the instrumentation out of a service in the middle. Nothing errors. You get a different conclusion.
Break it
Three moves, and none of them produces an error anywhere.
- Drop the header at
checkout → payments. The panel goes from one trace to two, both complete, both plausible, neither containing the other. In production this is a proxy that strips unknown headers, a client that rebuilds the request, or a queue that carries only the payload. - Take the instrumentation out of
payments. It still forwards the header, soledgerattaches tocheckout— its grandparent. The tree closes with one hop missing and still looks complete, and the time of the vanished hop is attributed to whoever is above it. - Let downstream services decide sampling on their own, then drop head sampling to 50%. You do not get more data: you get fragments, half-trees that look like evidence. That is why the decision travels in the header, and why the default sampler respects the one that arrived.
Predict before you look
Answer first, then run it.
A proxy between two of your services strips the traceparent header. What does the backend show?
Commit to an answer first. You cannot change it afterwards.
Instrument it
Two decisions from the file that actually runs in
labs/anatomy-of-a-trace/. The correct block is not written
here — it is lifted out of that file, so it cannot drift from the code
that compiles. Both wrong answers produce a panel you have already seen
above: one is the uninstrumented hop, the other is the stripped header.
Neither raises, logs, or fails a test that counts requests.
One service in the chain. The header that arrived is already extracted into `recebido`, and the SERVER span for this hop is already open in `span` — unless this instance is running uninstrumented, in which case `span` is null.
Which context becomes current for the rest of this request?
Span span =
semInstrumentacao
? null
: tracer.spanBuilder("GET /" + nome).setSpanKind(SpanKind.SERVER).setParent(recebido).startSpan();
Pick one. You cannot change it afterwards — and that is the point.
labs/anatomy-of-a-trace/app/src/main/java/anatomy/Service.java · OpenTelemetry Java 1.65.0
The same service, one step later. `atual` holds the context with this hop's span in it, and `pedido` is the outgoing request being built for the next service.
What gets written into the outgoing request's headers?
Pick one. You cannot change it afterwards — and that is the point.
labs/anatomy-of-a-trace/app/src/main/java/anatomy/Service.java · OpenTelemetry Java 1.65.0
Read the theory
A trace is a tree nobody owns is the prose version of this lab: no allocator, no completeness signal, and a structure that degrades instead of failing. For what carries the edge across a process boundary, read Context is the product.
Check your understanding
The assessment for this lab has not been authored yet.