Pillar — Observability

Prefix: OBS- What it measures: whether agents can see what they changed — structured logging, error aggregation, distributed tracing, health endpoints, metrics, and SLOs.

Agents that can't see what they changed can't learn from what they changed. A codebase with structured logging, distributed tracing, and error aggregation gives agents a feedback loop: change, deploy (or test), observe, learn. Without observability, agents degrade into open-loop systems — write code, ship, hope.

Criteria in this pillar

OBS-010 — Structured logging

Level: 2 · Scope: application · Check: deterministic
The application uses a structured logger (pino, consola, structlog, zerolog) emitting JSON logs with fields for level, timestamp, message, and request identifier.
Rationale: agents debugging production issues depend on queryable logs; ad-hoc console.log doesn't scale.
Evidence expected: logger dependency present; usage across the codebase; no raw console.* in prod paths.

OBS-020 — Error aggregation

Level: 3 · Scope: application · Check: deterministic
Server and client errors are reported to a structured error-aggregation service with stable error identifiers, searchable by time range and stack trace. Source maps are uploaded for client errors so stack traces resolve to source-level frames. This criterion is vendor-neutral — conforming implementations may use any structured error-aggregation service that meets the evidence requirements below.
Rationale: errors that never reach an agent's view cannot be fixed by an agent.
Evidence expected: an error-aggregation SDK or log-drain configuration is present in the application manifest; initialization is called server-side and client-side where both apply; source-map upload is configured for client builds.

OBS-030 — Distributed tracing

Level: 3 · Scope: application · Check: deterministic
Request handlers are instrumented to emit distributed traces, and spans propagate the active trace identifier into structured log entries so logs and traces correlate. Traces are exported to a tracing or APM backend that supports trace-by-ID lookup, latency percentile views, and error-tagged spans. This criterion is vendor-neutral — conforming implementations may use any tracing/APM backend (self-hosted or managed) that ingests OpenTelemetry-compatible traces and meets the evidence requirements below.
Rationale: cross-service causality is opaque without traces; agents cannot debug performance regressions without them.
Evidence expected: a tracing instrumentation library (OpenTelemetry SDK or an OTel-compatible equivalent) is present in the application manifest; instrumentation is registered at process start; an exporter or collector configuration targets a tracing/APM backend; trace IDs appear in log records emitted under OBS-010.

OBS-040 — Health endpoint

Level: 2 · Scope: application · Check: deterministic
/api/health (or equivalent) returns { status, version, timestamp } with HTTP 200 when healthy.
Rationale: uptime monitors, deploy gates, and agent-driven canary rollouts all depend on a stable health signal.
Evidence expected: route handler present; CI smoke-tests it post-build.

OBS-050 — Metrics exported

Level: 4 · Scope: application · Check: deterministic
Business and system metrics (request rate, error rate, latency p50/p95/p99) exported to a metrics backend supporting Prometheus-compatible or OpenTelemetry export. This criterion is vendor-neutral — conforming implementations may use any metrics backend that meets the evidence requirements below.
Rationale: agents reasoning about reliability need numeric signal; logs alone are too noisy.
Evidence expected: metrics exporter configured; dashboards linked from runbook.

OBS-060 — SLOs and alerting

Level: 5 · Scope: application · Check: manual
Service-level objectives are documented, measured, and trigger alerts at error-budget-burn thresholds.
Rationale: SLO discipline is the language agents need to reason about "is it safe to ship this change?" at scale.
Evidence expected: SLO document; alert config; error-budget tracking.

The 8 pillars — sibling pillars in v1.0.
Criteria catalog — all criteria, one page.
Maturity levels — how criteria combine into a level.

Criteria in this pillar

Level: 2 · Scope: application · Check: deterministic

The application uses a structured logger (pino, consola, structlog, zerolog) emitting JSON logs with fields for level, timestamp, message, and request identifier.

Rationale: agents debugging production issues depend on queryable logs; ad-hoc console.log doesn't scale.

Evidence expected: logger dependency present; usage across the codebase; no raw console.* in prod paths.

Level: 3 · Scope: application · Check: deterministic

Server and client errors are reported to a structured error-aggregation service with stable error identifiers, searchable by time range and stack trace. Source maps are uploaded for client errors so stack traces resolve to source-level frames. This criterion is vendor-neutral — conforming implementations may use any structured error-aggregation service that meets the evidence requirements below.

Rationale: errors that never reach an agent's view cannot be fixed by an agent.

Evidence expected: an error-aggregation SDK or log-drain configuration is present in the application manifest; initialization is called server-side and client-side where both apply; source-map upload is configured for client builds.

Level: 3 · Scope: application · Check: deterministic

Request handlers are instrumented to emit distributed traces, and spans propagate the active trace identifier into structured log entries so logs and traces correlate. Traces are exported to a tracing or APM backend that supports trace-by-ID lookup, latency percentile views, and error-tagged spans. This criterion is vendor-neutral — conforming implementations may use any tracing/APM backend (self-hosted or managed) that ingests OpenTelemetry-compatible traces and meets the evidence requirements below.

Rationale: cross-service causality is opaque without traces; agents cannot debug performance regressions without them.

Evidence expected: a tracing instrumentation library (OpenTelemetry SDK or an OTel-compatible equivalent) is present in the application manifest; instrumentation is registered at process start; an exporter or collector configuration targets a tracing/APM backend; trace IDs appear in log records emitted under OBS-010.

Level: 2 · Scope: application · Check: deterministic

/api/health (or equivalent) returns { status, version, timestamp } with HTTP 200 when healthy.

Rationale: uptime monitors, deploy gates, and agent-driven canary rollouts all depend on a stable health signal.

Evidence expected: route handler present; CI smoke-tests it post-build.

Level: 4 · Scope: application · Check: deterministic

Business and system metrics (request rate, error rate, latency p50/p95/p99) exported to a metrics backend supporting Prometheus-compatible or OpenTelemetry export. This criterion is vendor-neutral — conforming implementations may use any metrics backend that meets the evidence requirements below.

Rationale: agents reasoning about reliability need numeric signal; logs alone are too noisy.

Evidence expected: metrics exporter configured; dashboards linked from runbook.

Level: 5 · Scope: application · Check: manual

Service-level objectives are documented, measured, and trigger alerts at error-budget-burn thresholds.

Rationale: SLO discipline is the language agents need to reason about "is it safe to ship this change?" at scale.

Evidence expected: SLO document; alert config; error-budget tracking.