Investigation Playbooks
Customer Flow Broken
Use this playbook when a user-facing path is broken and you need to trace impact across services, requests, logs, and releases.
Situation and outcome
Use this playbook when a user-facing path is broken and you need to trace impact across services, requests, logs, and releases.
A customer reports a broken flow but the failing service is unclear. Several services show symptoms during the same user-facing action. You need a fast path from product symptom to technical evidence.
By the end of this playbook, you should translate the customer symptom into a concrete service path and identify the first service, endpoint, or dependency where the flow stops behaving correctly.
Evidence to collect
- Flow boundary
- Write down the customer action, expected result, reported time, and environment before choosing filters.
- Service path
- Use the service map and trace detail together: the map shows likely relationships, while the trace proves one request path.
- Request context
- Pair the failed or slow trace with logs from the same time window so the handoff includes both timing and application events.
Investigation path
Follow the path in order until the evidence points to a service, dependency, release, runtime signal, or setup gap.
- 1
Start in Services and identify services with error, latency, or traffic changes in the reported time range.
- 2
Use the service map to follow callers and callees around the suspected service.
- 3
Open Traces and filter to the most likely service, route, status, and time window.
- 4
Open one failed or slow trace and follow the span path across services.
- 5
Use related Logs to add event context around the selected trace.
- 6
Check Releases and Issues if the flow started failing after a recent change.
Decision point
If one service is the first failed span, use service detail to inspect its endpoints and dependencies. If the request reaches several services successfully before failing, assign the failing downstream segment. If no traces match the report, check service naming, environment filters, and time range.
Choose the next action only after the evidence explains both impact and ownership. If the evidence is stale, mismatched, or filtered too narrowly, widen the investigation before assigning work.
Handoff
A useful handoff gives the next responder enough context to reproduce the evidence without replaying the whole investigation.
Evidence 1
Describe the user-facing flow in one sentence and name the suspected service path.
Evidence 2
Include the first failed or slow span, plus the upstream caller and downstream dependency if relevant.
Evidence 3
List filters used during the investigation so another teammate can reproduce the view.
Evidence 4
Say explicitly when evidence is missing because traces, logs, or service identity need setup fixes.