An Observability Instrumentation Prompt That Covers Your Service Code

A service goes slow in production and the dashboard shows nothing, because the slow part was never instrumented. Adding spans, metrics, and logs after the fact is tedious, easy to do unevenly, and the kind of work that gets skipped under deadline. An observability instrumentation prompt makes it repeatable: it reads your function, adds the spans and structured logs that matter, and hands back a coverage checklist of what it touched and what it deliberately left alone.

Search this and almost everything is about instrumenting AI systems, tracing LLM calls and token usage. That's a real topic, but it's not this one. This is the older job of making your own service code observable, and there's surprisingly little that gives you a reusable prompt for it instead of a framework tutorial.

Instrumentation is a coverage problem

Good instrumentation isn't "add logging everywhere". It's deciding what to measure and covering it consistently. An external HTTP call deserves a span with timing and status. A pure function that adds two numbers does not. The skill is the line between them, and that line is exactly what a coverage contract encodes.

Give the prompt a contract that names the things worth a span (calls that leave the process, error branches, retry loops, anything touching IO) and the things that aren't (trivial accessors, side-effect-free helpers), and it instruments evenly. Skip the contract and you get one of two failures: every line wrapped in a span until the trace is unreadable, or three spans on a 200-line file that miss the slow path entirely.

The take: over-instrumentation is worse than under-instrumentation. A trace where everything is a span tells you nothing, because the signal is buried in noise you pay to store and query. Instrument the boundaries, not the internals. A prompt that respects a coverage contract beats one that tries to be thorough.

What you can do with this prompt

• Add spans around every external call and error path in a service function
• Emit structured logs with consistent field names instead of free-text strings
• Add the counters and histograms that matter (request rate, error rate, latency)
• Produce a coverage checklist of what got instrumented and what was skipped, with reasons
• Keep instrumentation consistent across files written by different people
• Flag code that's untestable for observability and explain why

Anatomy of the instrumentation prompt

Variables → Prompt → Output

{{source_code}}        the function or file to instrument
{{telemetry_stack}}    OpenTelemetry, your logging lib, metric names
{{span_naming}}        your convention (e.g. service.operation)
{{coverage_rules}}     what deserves a span vs what doesn't

→ Coverage contract + instrumentation patterns →

→ Output:
   1. Instrumented code (same logic, added telemetry)
   2. ## Coverage checklist
      Instrumented: <list with reason>
      Skipped: <list with reason>
      Needs human review: <ambiguous cases>

The coverage checklist is the part most tutorials skip and the part that makes this auditable. Code that comes back with new spans but no record of what was skipped leaves you guessing whether the slow path got covered. The checklist closes that gap.

Step-by-step usage

1. Gather inputs

Paste the function or file into {{source_code}}. Name your telemetry stack and metric conventions in {{telemetry_stack}} so the prompt uses your library, not a generic one. Specify {{span_naming}} so the spans match your existing traces.

2. Fill variables

Fill {{coverage_rules}} with your real policy. If you don't have one written down, the act of filling this variable is the moment you write it. That's a feature, not overhead.

3. Run the prompt

A low to moderate temperature works here. You want deterministic instrumentation decisions but the code generation has a little room. Temperature 0.2 to 0.4 is a reasonable band.

4. Post-process

Read the coverage checklist before the code. If the prompt skipped a path you care about, the {{coverage_rules}} were too narrow. Check that span names match your convention exactly, because a near-miss naming breaks trace grouping silently.

5. Iterate

If the instrumentation is too dense, tighten the "skip" rules. If it missed the slow loop, add "instrument any loop over an external resource" to the coverage rules and rerun.

Prompt-craft patterns that matter here

The skip-with-reason rule. Force the prompt to explain every skip. A span that's missing because the prompt judged it unnecessary is fine if it told you. A silent omission is a future blind spot.

For every function or branch you do not instrument, record it under
"Skipped" with a one-line reason. Never silently omit a path. Loops over
external resources and error branches are always instrumented or flagged.

The convention-binding. Pass your span-naming convention as a variable and the prompt produces traces that group with your existing ones. Leave it out and the model invents names that look right but don't match, so the new spans float disconnected from your dashboards.

Variables you'll set

Getting started

1. Paste the function into {{source_code}} and name your telemetry stack.
2. Set {{span_naming}} so spans match your existing traces.
3. Write your {{coverage_rules}}, even a rough version.
4. Run the prompt at a low to moderate temperature.
5. Read the coverage checklist before the instrumented code.
6. Confirm span names match your convention exactly, then commit.
7. For instrumenting a whole service consistently, start from the Observability Instrumentation Harness.

Instrumenting one function with a copied prompt is fine. Instrumenting a service so every file follows the same coverage contract is where the harness earns it, because the contract and the skip-with-reason rule stay consistent across every engineer who runs it.

Better instrumentation is what makes the next incident debuggable, so this pairs with the blameless incident postmortem prompt. The drift you catch in infra often shows up first as a span anomaly, which is why the terraform drift detection prompt is a natural companion, and the Infrastructure Drift Detection Harness ships that contract ready to run.