Testland
Browse all skills & agents

zipkin-trace-tests

Author integration tests that query Zipkin for trace verification - Zipkin all-in-one Docker for CI, REST API (`/api/v2/traces`, `/api/v2/services`, `/api/v2/dependencies`), B3 propagation header tests (single-header and multi-header X-B3-* form), dependency-graph assertions. Use when the team uses Zipkin (legacy or Spring Cloud Sleuth heritage).

Install with skills.sh (any agent)

npx skills add testland/qa --skill zipkin-trace-tests
View source

zipkin-trace-tests

Zipkin is the original distributed-tracing system (predates OpenTelemetry); still common in Java shops via Spring Cloud Sleuth heritage. Per the Zipkin quickstart docs (opens in new window), a single Docker command provides the full server + UI.

When to use

  • Existing Spring Cloud Sleuth / Spring Boot Actuator setup uses Zipkin as the trace store.
  • Team standardized on B3 header propagation (vs W3C Trace Context).
  • Migrating from Zipkin to Jaeger / OpenTelemetry - these tests protect during cutover.

Step 1 - Run Zipkin in CI

Per the Zipkin quickstart docs (opens in new window):

docker run -d -p 9411:9411 openzipkin/zipkin

GitHub Actions service:

services:
  zipkin:
    image: openzipkin/zipkin
    ports:
      - 9411:9411

Step 2 - REST API endpoints

Per the Zipkin API spec (opens in new window):

EndpointReturns
GET /api/v2/servicesList of service names
GET /api/v2/spans?serviceName=XList of operations
GET /api/v2/traces?serviceName=X&spanName=Y&lookback=300000&limit=10List of traces
GET /api/v2/trace/{traceId}Single trace
GET /api/v2/dependencies?endTs=...&lookback=...Service dependency graph
POST /api/v2/spansSubmit spans (V2 JSON)

lookback is in milliseconds.

Step 3 - Configure SDK to ship to Zipkin

from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.zipkin.json import ZipkinExporter

provider = TracerProvider()
provider.add_span_processor(
    BatchSpanProcessor(ZipkinExporter(endpoint="http://localhost:9411/api/v2/spans"))
)
trace.set_tracer_provider(provider)

OTLP ingest is also possible via Zipkin's compatibility port (port 9411 also accepts B3-format JSON via OTLP-to-Zipkin bridges).

Step 4 - Query trace + assert shape

import requests, time
from opentelemetry import trace

def test_order_trace_in_zipkin():
    with use_tracer():
        create_order(items=[item])

    trace.get_tracer_provider().force_flush(timeout_millis=5000)
    time.sleep(0.5)

    resp = requests.get(
        "http://localhost:9411/api/v2/traces",
        params={"serviceName": "orders", "spanName": "order.create",
                "lookback": 60000, "limit": 1},
    )
    traces = resp.json()  # list of lists of spans
    assert len(traces) == 1
    spans = traces[0]
    create_span = next(s for s in spans if s["name"] == "order.create")
    assert create_span["tags"]["order.item_count"] == "1"  # tags are strings in Zipkin V2

Note Zipkin V2 stores tag values as strings (vs Jaeger's typed tags). Cast in assertions accordingly.

Step 5 - B3 propagation header tests

Per the B3 propagation spec (opens in new window):

Multi-header form:

  • X-B3-TraceId: 32 or 16 lower-hex characters
  • X-B3-SpanId: 16 lower-hex characters
  • X-B3-ParentSpanId: 16 lower-hex characters (absent on root)
  • X-B3-Sampled: 1 (accept) or 0 (deny)
  • X-B3-Flags: 1 for debug

Single-header form:

b3: {TraceId}-{SpanId}-{SamplingState}-{ParentSpanId}

Example:

b3: 80f198ee56343ba864fe8b2a57d3eff7-e457b5a2e4d86bd1-1-05e3ac9a4f6e3b90

Per the B3 propagation spec (opens in new window), sampling values: 1 accept, 0 deny, d debug, absent = defer.

Test both forms:

def test_b3_single_header_propagates():
    headers = {"b3": f"{trace_id}-{span_id}-1-{parent_id}"}
    resp = requests.get("http://localhost:8080/orders", headers=headers)

    # Verify downstream service preserved trace_id
    time.sleep(0.5)
    traces = requests.get(
        "http://localhost:9411/api/v2/trace/" + trace_id
    ).json()
    assert any(s["traceId"] == trace_id for s in traces)

Step 6 - Dependency graph assertion

Zipkin computes service dependencies from observed traces:

def test_orders_calls_payments():
    # Exercise the cross-service flow
    create_order_with_payment()
    trace.get_tracer_provider().force_flush(5000)
    time.sleep(2.0)  # dependency aggregation can be lazy

    end_ts = int(time.time() * 1000)
    deps = requests.get(
        "http://localhost:9411/api/v2/dependencies",
        params={"endTs": end_ts, "lookback": 60000},
    ).json()

    pair = next(
        (d for d in deps if d["parent"] == "orders" and d["child"] == "payments"),
        None,
    )
    assert pair is not None
    assert pair["callCount"] >= 1

Note dependency calculation is lazy + aggregated - may need a short delay or explicit dependency-aggregation trigger depending on storage backend. In-memory storage computes inline; Cassandra backend uses Spark batch.

Step 7 - Per-test isolation

service_name = f"orders-test-{uuid4()}"
# ... use this service_name in SDK config + Zipkin queries ...

Same pattern as Jaeger; Zipkin in-memory storage is bounded.

Anti-patterns

Anti-patternWhy it failsFix
Query immediately after exerciseSpans batch-shipped; not yet in Zipkinforce_flush + sleep (Step 4)
Assert tag values as integersZipkin V2 tags are all stringsCast or compare as string (Step 4)
Test only multi-header B3Modern services use single-header formTest both forms (Step 5)
Expect dependency-graph immediatelyAggregation is lazy on most backendsAllow ≥2s delay (Step 6)
Hard-code production Zipkin URL in testsTest traces pollute prodDocker all-in-one only (Step 1)

Limitations

  • Zipkin V2 JSON does not natively carry span.kind as an enumerated field; encoded as kind string ("CLIENT" / "SERVER" / etc.). Cross-check the Zipkin API spec (opens in new window) for current schema.
  • B3 multi-header is being phased out by some libraries in favor of single-header b3:. Track per-language SDK behavior.
  • Zipkin's storage retention varies per backend; in-memory evicts on pressure.

References

Related skills

jaeger-trace-tests

Author integration tests that query Jaeger for cross-service trace verification - Jaeger all-in-one Docker for CI (OTLP gRPC :4317 + HTTP :4318 ingest, query API on :16686), `/api/traces?service=X&operation=Y` query patterns, span set + parent-child + duration assertions. Use when verifying that a request produces the expected spans across service boundaries in a running Jaeger backend.

opentelemetry-trace-assertions

Author trace-shape assertions in tests using OpenTelemetry SDK in-memory exporter - capture spans during test execution, assert on span name + attributes + status + parent-child structure + duration. Cross-language patterns (Python `InMemorySpanExporter` + `SimpleSpanProcessor`, JS `getRecordedSpans()`, Java `OpenTelemetryExtension`); CI integration. Use when a service is instrumented with the OpenTelemetry SDK and downstream alerts, SLOs, or dashboards depend on specific span names or attributes that a refactor could silently drop.

otel-collector-config-tester

Validates OpenTelemetry Collector pipeline configurations and verifies spans flow end-to-end through the collector: runs `otelcol validate --config`, wires the `debug`/`file` exporter for span-output assertions, and integrates the full cycle into CI. Use when a collector config change (new receiver, processor swap, exporter wiring) needs correctness verification before deployment.

tempo-trace-tests

Authors integration tests that query Grafana Tempo for cross-service trace verification - TraceQL `{ }` span selectors targeting `span.`, `resource.`, and intrinsic fields; Tempo HTTP API (`/api/search` with `q=`, `/api/traces/{id}`) for span-set and attribute assertions; local Tempo via Docker single-binary (ports 4317/4318/3200). Use when the production observability stack uses Tempo as the trace backend and tests must verify distributed trace shape, span attributes, or service topology after instrumentation changes.

trace-spec-author

Build a trace specification document per feature - defines the trace shape (root span + child spans + key attributes per OpenTelemetry semantic conventions) that production code MUST emit. The spec drives both implementation reviews AND trace-assertion tests, so a single declarative document is the source of truth for what observability "looks like" for a feature. Use before instrumenting a new feature, when existing spans have grown organically with no agreed shape, or after an incident where a debugging session stalled on missing span attributes.