The criteria catalog is versioned independently from the standard. This page renders the v1.0 catalog verbatim. For the source-of-truth file, see spec/v1.0/criteria-catalog.md.

CARL Standard v1.0 — Criteria Catalog

How to read this catalog

Each criterion has a stable ID prefixed by its pillar:

• Scope is repository, application, or organization per §4.3 of the standard.
• Check type is deterministic, ai-assisted, or manual. The standard requires ≥80% deterministic across the catalog — this v1.0 catalog satisfies that.
• Evidence expected describes what an assessor looks for. Sanitized for reports: structural observations, file paths, config keys. Never raw code.

Criteria at level N must be met to claim level N. To be assigned level N, a subject must pass ≥80% of criteria at level N and all prior levels, applicability-adjusted.

Pillar 1 — Documentation & Knowledge

DOC-010: README present and substantive

• Pillar: Documentation & Knowledge
• Level: 0
• Scope: repository
• Check type: deterministic
• Description: A README.md (or README variant) exists at the repository root and is longer than 20 non-blank lines, referencing at minimum the project purpose, setup command, and primary development command.
• Rationale: Agents spawned into a repository read the README first to build a mental model; without one, every task begins with ambiguity and costly discovery.
• Evidence expected: File exists at repo root; word count, section headings detected.

DOC-020: Project purpose stated in README

• Pillar: Documentation & Knowledge
• Level: 1
• Scope: repository
• Check type: ai-assisted
• Description: The README contains a clear, one-paragraph description of what the project does and for whom.
• Rationale: Agents that understand intent make better local decisions — they refactor with goals in mind rather than preserving incidental structure.
• Evidence expected: Section heading detected plus AI-assisted judgment of whether the paragraph answers "what does this do and why."

DOC-030: AGENTS.md or CLAUDE.md present

• Pillar: Documentation & Knowledge
• Level: 1
• Scope: repository
• Check type: deterministic
• Description: A canonical agent-context file (AGENTS.md, CLAUDE.md, or .cursor/rules) is present at the repo root and references stack, conventions, and common commands.
• Rationale: Agents behave better when given explicit rules than when inferring conventions from scattered files. This is the single highest-leverage artifact for agent readiness.
• Evidence expected: File exists; contains sections matching ^##?\s*(Stack|Conventions|Commands|Setup).

DOC-040: Environment variables documented

• Pillar: Documentation & Knowledge
• Level: 1
• Scope: application
• Check type: deterministic
• Description: Every environment variable the application requires is documented in docs/env.md, README.md, or an equivalent table, and a committed .env.example exists.
• Rationale: Agents cannot guess undocumented secrets or URLs; missing env documentation blocks every subsequent task.
• Evidence expected: .env.example presence; ratio of env.* references in code to documented entries.

DOC-050: Architecture Decision Records folder

• Pillar: Documentation & Knowledge
• Level: 2
• Scope: repository
• Check type: deterministic
• Description: An docs/adr/ (or docs/decisions/) folder exists with at least a template and one recorded decision.
• Rationale: ADRs encode why — agents use them to avoid re-litigating settled questions and to respect hard constraints with non-obvious rationale.
• Evidence expected: Directory present; at least one NNNN-*.md ADR file.

DOC-060: Runbook / operations documentation

• Pillar: Documentation & Knowledge
• Level: 2
• Scope: application
• Check type: ai-assisted
• Description: A docs/runbook.md (or equivalent) describes on-call procedures, common failure modes, dashboards, and rollback steps.
• Rationale: Agents that can read runbooks can assist during incidents; those that cannot are dead weight at the worst moments.
• Evidence expected: File presence; AI-assisted judgment on content coverage.

DOC-070: API reference auto-generated or hand-maintained

• Pillar: Documentation & Knowledge
• Level: 3
• Scope: application
• Check type: deterministic
• Description: API (REST, gRPC, GraphQL, or Server Actions) has a machine-readable specification or hand-maintained reference that is up-to-date with the code.
• Rationale: Agents consuming or extending an API benefit from a typed, machine-readable contract; without it, every change risks silent contract breakage.
• Evidence expected: openapi.yaml, openapi.json, schema.graphql, or similar; or a published reference site built from code.

DOC-080: Changelog in Keep-a-Changelog format

• Pillar: Documentation & Knowledge
• Level: 2
• Scope: repository
• Check type: deterministic
• Description: CHANGELOG.md follows Keep-a-Changelog format with semver-aligned entries.
• Rationale: Release history gives agents (and humans) a searchable record of what changed when, reducing regression-hunting time.
• Evidence expected: File presence; headings match ## [X.Y.Z] - YYYY-MM-DD.

Pillar 2 — Code Quality & Validation

CQV-010: Linter configured and runnable

• Pillar: Code Quality & Validation
• Level: 0
• Scope: application
• Check type: deterministic
• Description: A linter is configured for the primary language (ESLint, Ruff, golangci-lint, Clippy, etc.) and runs via pnpm lint (or equivalent) with a clean exit on current main.
• Rationale: Agents produce cleaner output when they can check their work locally before committing; without a linter, style drifts and review cost rises.
• Evidence expected: Config file present; package.json.scripts.lint (or equivalent) defined.

CQV-020: Formatter configured with a single source of truth

• Pillar: Code Quality & Validation
• Level: 1
• Scope: repository
• Check type: deterministic
• Description: A formatter (Prettier, Black, gofmt, rustfmt) has a project config and runs via a format / format:check command. .editorconfig present.
• Rationale: Style disputes consume review cycles agents cannot resolve. A formatter shifts the debate out of PR reviews.
• Evidence expected: .prettierrc / pyproject.toml / rustfmt.toml; .editorconfig.

CQV-030: TypeScript/typed language in strict mode

• Pillar: Code Quality & Validation
• Level: 1
• Scope: application
• Check type: deterministic
• Description: For typed languages, strict mode is enabled (e.g., TypeScript strict: true, noUncheckedIndexedAccess: true; mypy strict = true). Type errors block CI.
• Rationale: Strict types catch 20% of bugs agents would otherwise ship — and provide a signal agents can use to ground refactors.
• Evidence expected: tsconfig.json.compilerOptions.strict === true and related flags; mypy/ruff settings.

CQV-040: Pre-commit hooks enforce formatter and linter

• Pillar: Code Quality & Validation
• Level: 1
• Scope: repository
• Check type: deterministic
• Description: Husky + lint-staged (or pre-commit for Python) is configured to run format and lint on staged files.
• Rationale: Local enforcement prevents broken commits from reaching CI, shortening the feedback loop for agents.
• Evidence expected: .husky/pre-commit or .pre-commit-config.yaml present and referenced in setup.

CQV-050: No forbidden escape hatches in production code

• Pillar: Code Quality & Validation
• Level: 2
• Scope: application
• Check type: deterministic
• Description: Source contains no @ts-ignore, @ts-nocheck, // eslint-disable-next-line (without rationale + tracking issue), # type: ignore, or equivalent silencers outside test fixtures.
• Rationale: Escape hatches accumulate into silent bugs agents cannot detect. A policy of "none without a tracked reason" keeps signal high.
• Evidence expected: Grep counts; flagged sites require inline rationale comment and ticket reference.

CQV-060: No raw console.log / print in production paths

• Pillar: Code Quality & Validation
• Level: 2
• Scope: application
• Check type: deterministic
• Description: Production code uses the project's logger; raw console.log / print / fmt.Println calls are absent or lint-gated.
• Rationale: Structured logs are queryable; ad-hoc prints are not. Agents debugging production incidents depend on structure.
• Evidence expected: ESLint rule no-console configured; grep reports zero violations.

CQV-070: Import ordering enforced

• Pillar: Code Quality & Validation
• Level: 2
• Scope: application
• Check type: deterministic
• Description: Imports are grouped and ordered consistently (external → internal, alphabetical). Enforced by ESLint or equivalent.
• Rationale: Consistency reduces agent cognitive load on every read; diff noise shrinks.
• Evidence expected: eslint-plugin-import or equivalent configured with ordering rules.

CQV-080: Code complexity gates

• Pillar: Code Quality & Validation
• Level: 3
• Scope: application
• Check type: deterministic
• Description: CI measures and fails on thresholds for cyclomatic complexity, file length, or cognitive complexity (e.g., SonarCloud, eslint-plugin-sonarjs, radon).
• Rationale: Complex functions are where agents introduce the most subtle bugs. Gating on complexity limits the blast radius.
• Evidence expected: Complexity config present in CI; at least one threshold defined and enforced.

CQV-090: CI enforces a full quality gate (tests + lint and/or build)

• Pillar: Code Quality & Validation
• Level: 4
• Scope: application
• Check type: deterministic
• Description: CI enforces a full quality gate — the test suite plus lint and/or build — on changes.
• Rationale: An agent can only drive a scoped task end-to-end when an automated gate proves its change is green before merge; without one a human must hand-verify every edit.
• Evidence expected: CI workflow inspection; a single gate runs the test suite and the linter and/or build on every push and PR.

Pillar 3 — Testing

TST-010: Test framework present

• Pillar: Testing
• Level: 0
• Scope: application
• Check type: deterministic
• Description: A test framework is configured (Vitest, Jest, pytest, Go test, etc.) and at least one test exists and passes.
• Rationale: Agents can only verify their own changes if tests exist to run.
• Evidence expected: Test runner config; package.json.scripts.test; at least one green test.

TST-020: Coverage measurement configured

• Pillar: Testing
• Level: 1
• Scope: application
• Check type: deterministic
• Description: Coverage tooling runs in CI and emits text/html/lcov output. A coverage command exists (pnpm test:coverage).
• Rationale: Coverage is a crude but essential signal for agents choosing where to add tests.
• Evidence expected: Vitest coverage config or equivalent; coverage artifact in CI output.

TST-030: Coverage thresholds enforced at ≥ 80%

• Pillar: Testing
• Level: 2
• Scope: application
• Check type: deterministic
• Description: CI fails if coverage drops below 80% on branches, functions, lines, and statements.
• Rationale: Without enforcement, coverage metrics drift downward as features are added without tests.
• Evidence expected: vitest.config.* or equivalent with thresholds: { branches: 80, functions: 80, lines: 80, statements: 80 }.

TST-040: End-to-end tests for critical journeys

• Pillar: Testing
• Level: 2
• Scope: application
• Check type: deterministic
• Description: At least one Playwright (or equivalent) E2E test exists covering signup/login (if applicable) and the core feature happy path.
• Rationale: Agents cannot simulate browser interactions themselves; E2E coverage provides a verification layer they depend on.
• Evidence expected: playwright.config.* + at least one *.spec.ts under e2e/ or tests/e2e/.

TST-050: Resilient selectors in UI tests

• Pillar: Testing
• Level: 2
• Scope: application
• Check type: ai-assisted
• Description: UI tests use data-testid, role, or accessible label selectors rather than CSS classes or tag chains.
• Rationale: CSS-selector-based tests break on styling refactors, which agents perform frequently. Semantic selectors remain stable.
• Evidence expected: Grep on test files for className-style selectors vs getByRole / getByTestId; AI-assisted judgment on ratio.

TST-060: Test isolation (no shared mutable state)

• Pillar: Testing
• Level: 3
• Scope: application
• Check type: ai-assisted
• Description: Tests do not depend on execution order or shared database state. DB tests use transactions or per-test schemas.
• Rationale: Flaky tests poison the signal agents rely on to verify their changes.
• Evidence expected: Evidence of transaction setup/teardown; AI-assisted judgment; flake rate metric if collected.

TST-070: Flake detection and quarantine

• Pillar: Testing
• Level: 3
• Scope: application
• Check type: deterministic
• Description: CI detects flaky tests (via retry + failure-rate measurement) and tracks them in an issue or dashboard.
• Rationale: Flakes are the silent killer of agent-driven delivery. Detection is the prerequisite for remediation.
• Evidence expected: Configured retry + reporter; issue tracker or dashboard references.

TST-080: Property-based or fuzz testing for critical modules

• Pillar: Testing
• Level: 4
• Scope: application
• Check type: manual
• Description: Critical parsers, validators, or serializers have property-based (fast-check, hypothesis) or fuzz tests.
• Rationale: Example-based tests miss the adversarial inputs agents must handle. PBT raises the floor.
• Evidence expected: Import of fast-check / hypothesis / go-fuzz; at least one property test.

Pillar 4 — Build & Environment

BLD-010: Reproducible build

• Pillar: Build & Environment
• Level: 0
• Scope: application
• Check type: deterministic
• Description: pnpm build (or equivalent) succeeds without warnings on a clean clone and produces consistent output given the same inputs.
• Rationale: A broken or nondeterministic build blocks every agent task downstream.
• Evidence expected: Build command defined; green build on CI; no warnings.

BLD-020: Node/runtime version pinned

• Pillar: Build & Environment
• Level: 1
• Scope: repository
• Check type: deterministic
• Description: .nvmrc / .tool-versions / pyproject.toml.tool.python.requires / go.mod.go pins the runtime version. For Node, package.json.engines.node matches.
• Rationale: Version drift causes "works on my machine" bugs that agents cannot reproduce.
• Evidence expected: Pin file present; constraint narrow (e.g., >=22 <23).

BLD-030: Lockfile committed

• Pillar: Build & Environment
• Level: 0
• Scope: repository
• Check type: deterministic
• Description: pnpm-lock.yaml / poetry.lock / Cargo.lock / go.sum is committed.
• Rationale: Without a lockfile, dependency resolution is nondeterministic and bugs become unreproducible.
• Evidence expected: Lockfile present; not in .gitignore.

BLD-040: .env.example documents every env var

• Pillar: Build & Environment
• Level: 1
• Scope: application
• Check type: deterministic
• Description: A committed .env.example lists every env variable the application consumes, with placeholder values.
• Rationale: Agents setting up a fresh environment need a complete, authoritative list — not archaeology across the codebase.
• Evidence expected: .env.example present; count of vars ≥ count of process.env.* references in source.

BLD-050: Local-setup script or devcontainer

• Pillar: Build & Environment
• Level: 2
• Scope: application
• Check type: deterministic
• Description: A .devcontainer/devcontainer.json exists, OR a documented scripts/setup.sh takes a fresh clone to a working local environment in one command.
• Rationale: Agents in fresh environments cannot ask "how do I get this running"; they need a reproducible path.
• Evidence expected: Devcontainer config or setup script; documentation linking to it.

BLD-060: Hermetic CI (no network for test/build beyond cache)

• Pillar: Build & Environment
• Level: 3
• Scope: application
• Check type: manual
• Description: CI jobs declare their dependencies explicitly; no curl | bash steps; lockfile-based installs only; network egress reviewed.
• Rationale: Hermeticity removes a class of CI flake that masks real test failures.
• Evidence expected: CI workflow review; absence of curl | bash or untracked download steps.

BLD-070: Docker image produced and published

• Pillar: Build & Environment
• Level: 3
• Scope: application
• Check type: deterministic
• Description: A Dockerfile builds a working image; CI publishes it to a registry on release.
• Rationale: Portable runtime artifacts let agents reproduce the production environment locally.
• Evidence expected: Dockerfile present; release workflow includes image publish step.

BLD-080: Infrastructure provisioned as code — zero-to-prod is codified

• Pillar: Build & Environment
• Level: 5
• Scope: application
• Check type: deterministic
• Description: Infrastructure is provisioned as code (IaC), so the system can be stood up from scratch.
• Rationale: An agent that stands up a new business needs to provision the running environment deterministically; IaC is the machine-actionable form of "deploy from zero".
• Evidence expected: IaC artifacts detected (Terraform, Pulumi, CDK, SST, Serverless, Bicep, or Wrangler config).

Pillar 5 — Observability

OBS-010: Structured logging

• Pillar: Observability
• Level: 1
• Scope: application
• Check type: deterministic
• Description: The application uses a structured logger (pino, consola, structlog, zerolog) emitting JSON logs with fields for level, timestamp, message, and request identifier.
• Rationale: Agents debugging production issues depend on queryable logs; ad-hoc console.log doesn't scale.
• Evidence expected: Logger dependency present; usage across the codebase; no raw console.* in prod paths.

OBS-020: Error aggregation

• Pillar: Observability
• Level: 2
• Scope: application
• Check type: deterministic
• Description: Server and client errors are reported to a structured error-aggregation service with stable error identifiers, searchable by time range and stack trace. Source maps are uploaded for client errors so stack traces resolve to source-level frames. This criterion is vendor-neutral — conforming implementations may use any structured error-aggregation service that meets the evidence requirements below.
• Rationale: Errors that never reach an agent's view cannot be fixed by an agent.
• Evidence expected: An error-aggregation SDK or log-drain configuration is present in the application manifest; initialization is called server-side and client-side where both apply; source-map upload is configured for client builds.

OBS-030: Distributed tracing

• Pillar: Observability
• Level: 2
• Scope: application
• Check type: deterministic
• Description: Request handlers are instrumented to emit distributed traces, and spans propagate the active trace identifier into structured log entries so logs and traces correlate. Traces are exported to a tracing or APM backend that supports trace-by-ID lookup, latency percentile views, and error-tagged spans. This criterion is vendor-neutral — conforming implementations may use any tracing/APM backend (self-hosted or managed) that ingests OpenTelemetry-compatible traces and meets the evidence requirements below.
• Rationale: Cross-service causality is opaque without traces; agents cannot debug performance regressions without them.
• Evidence expected: A tracing instrumentation library (OpenTelemetry SDK or an OTel-compatible equivalent) is present in the application manifest; instrumentation is registered at process start; an exporter or collector configuration targets a tracing/APM backend; trace IDs appear in log records emitted under OBS-010.

OBS-040: Health endpoint

• Pillar: Observability
• Level: 1
• Scope: application
• Check type: deterministic
• Description: /api/health (or equivalent) returns { status, version, timestamp } with HTTP 200 when healthy.
• Rationale: Uptime monitors, deploy gates, and agent-driven canary rollouts all depend on a stable health signal.
• Evidence expected: Route handler present; CI smoke-tests it post-build.

OBS-050: Metrics exported

• Pillar: Observability
• Level: 3
• Scope: application
• Check type: deterministic
• Description: Business and system metrics (request rate, error rate, latency p50/p95/p99) exported to Prometheus, Datadog, or an OpenTelemetry metrics backend.
• Rationale: Agents reasoning about reliability need numeric signal; logs alone are too noisy.
• Evidence expected: Metrics exporter configured; dashboards linked from runbook.

OBS-060: SLOs and alerting

• Pillar: Observability
• Level: 4
• Scope: application
• Check type: manual
• Description: Service-level objectives are documented, measured, and trigger alerts at error-budget-burn thresholds.
• Rationale: SLO discipline is the language agents need to reason about "is it safe to ship this change?" at scale.
• Evidence expected: SLO document; alert config; error-budget tracking.

OBS-070: Product analytics / business KPIs are instrumented

• Pillar: Observability
• Level: 6
• Scope: application
• Check type: deterministic
• Description: Product analytics or business-KPI instrumentation (an analytics SDK) is present.
• Rationale: Operating a business autonomously requires a feedback signal beyond infra metrics — product/usage analytics is the instrumentation an agent steers on.
• Evidence expected: A product-analytics dependency (e.g. PostHog, Segment, Mixpanel) detected in a manifest.

Pillar 6 — Security & Access Control

SEC-010: No committed secrets

• Pillar: Security & Access Control
• Level: 0
• Scope: repository
• Check type: deterministic
• Description: gitleaks (or trufflehog) scan of the full git history returns zero findings of API keys, tokens, private keys, or passwords.
• Rationale: A leaked secret in history is a rotation emergency; agents must never author one.
• Evidence expected: .gitleaks.toml present; CI runs gitleaks; zero findings.

SEC-020: .env* gitignored

• Pillar: Security & Access Control
• Level: 0
• Scope: repository
• Check type: deterministic
• Description: .gitignore excludes .env, .env.local, .env.*.local, *.pem, *.key, service-account*.json, .aws/.
• Rationale: Default-deny on sensitive files is the cheapest single control.
• Evidence expected: .gitignore contents.

SEC-030: Dependency audit gate

• Pillar: Security & Access Control
• Level: 1
• Scope: repository
• Check type: deterministic
• Description: CI runs pnpm audit --prod (or equivalent) and fails on HIGH or CRITICAL findings.
• Rationale: Known-vulnerable dependencies in production are a preventable class of incident; agents pulling new deps must know the bar.
• Evidence expected: CI step; failure behavior confirmed.

SEC-040: CODEOWNERS present

• Pillar: Security & Access Control
• Level: 1
• Scope: repository
• Check type: deterministic
• Description: CODEOWNERS exists at repo root and assigns at least one owner to the entire tree.
• Rationale: Ownership routes review to the right person; unreviewed changes are the most common regression source.
• Evidence expected: CODEOWNERS file present; syntax valid.

SEC-050: Branch protection on main

• Pillar: Security & Access Control
• Level: 2
• Scope: repository
• Check type: manual
• Description: main requires pull request review, passing CI, and no force-push. Documented in SECURITY.md or inferable from GitHub API.
• Rationale: Without branch protection, a compromised agent or account can land unreviewed code.
• Evidence expected: GitHub branch protection rules; documentation acknowledging them.

SEC-060: DCO or equivalent contribution gating

• Pillar: Security & Access Control
• Level: 2
• Scope: repository
• Check type: deterministic
• Description: Every commit to main contains a Signed-off-by trailer (DCO), or repository policy requires a CLA. Enforced via CI check.
• Rationale: Contribution provenance matters for legal and trust reasons; machine-checkable enforcement is cheap.
• Evidence expected: Commits on main inspected; DCO action in CI.

SEC-070: Dependabot or Renovate configured

• Pillar: Security & Access Control
• Level: 2
• Scope: repository
• Check type: deterministic
• Description: .github/dependabot.yml or renovate.json configured for both application dependencies and GitHub Actions.
• Rationale: Automated dependency updates close the gap between vulnerability disclosure and patching.
• Evidence expected: Config file present; PRs visible in history.

SEC-080: Security headers set

• Pillar: Security & Access Control
• Level: 3
• Scope: application
• Check type: deterministic
• Description: The application sets Strict-Transport-Security, X-Content-Type-Options: nosniff, X-Frame-Options: DENY (or frame-ancestors CSP), Referrer-Policy, Permissions-Policy, and a restrictive Content-Security-Policy.
• Rationale: Headers are free mitigation for entire bug classes; agents touching edge config must preserve them.
• Evidence expected: Inspection of proxy.ts / next.config.*; response headers on deployed endpoints.

SEC-090: SBOM generated on release

• Pillar: Security & Access Control
• Level: 4
• Scope: application
• Check type: deterministic
• Description: A Software Bill of Materials (CycloneDX or SPDX) is generated and attached to each release, and optionally signed (Sigstore).
• Rationale: Supply-chain incidents (e.g., xz) cost more when you don't know what's in your artifacts.
• Evidence expected: SBOM artifact on a recent release; tooling in CI.

Pillar 7 — Task Intake & Routing

TIR-010: Issue templates present

• Pillar: Task Intake & Routing
• Level: 1
• Scope: repository
• Check type: deterministic
• Description: .github/ISSUE_TEMPLATE/ contains at least bug_report and feature_request templates.
• Rationale: Structured intake gives agents (triage, labeling, reproducing) enough consistent input to do meaningful work.
• Evidence expected: Directory present with at least two templates.

TIR-020: Pull request template

• Pillar: Task Intake & Routing
• Level: 1
• Scope: repository
• Check type: deterministic
• Description: .github/PULL_REQUEST_TEMPLATE.md exists with at minimum: summary, related issues, and a pre-merge checklist.
• Rationale: PR hygiene is one of the signals most strongly correlated with a healthy review culture.
• Evidence expected: File present; sections for summary, linking, checklist.

TIR-030: Labels defined and documented

• Pillar: Task Intake & Routing
• Level: 2
• Scope: repository
• Check type: deterministic
• Description: Labels are defined in .github/labels.yml (or the repo's label set is documented) and synced to GitHub.
• Rationale: Consistent labels let agents filter, route, and prioritize — an ad-hoc label set makes automation unreliable.
• Evidence expected: labels.yml present; label-sync action configured.

TIR-040: Triage convention documented

• Pillar: Task Intake & Routing
• Level: 3
• Scope: organization
• Check type: manual
• Description: A documented triage process exists (owner, cadence, SLAs) and is followed — recent issues show triage labels applied within the SLA window.
• Rationale: Untriaged backlogs are where agent-delegated work goes to die.
• Evidence expected: Process document; issue label timestamps.

TIR-050: Structured task intake — issue / PR templates an agent can fill

• Pillar: Task Intake & Routing
• Level: 4
• Scope: repository
• Check type: deterministic
• Description: Structured task-intake templates (issue and/or pull-request templates) are present.
• Rationale: Templates give an agent a machine-readable contract for what a task or change must contain, so intake is routable rather than free-form prose.
• Evidence expected: Issue and/or pull-request template files detected under .github/.

Pillar 8 — Delivery & Measurement

DLM-010: CI runs every gate on every PR

• Pillar: Delivery & Measurement
• Level: 1
• Scope: repository
• Check type: deterministic
• Description: .github/workflows/ci.yml (or equivalent) runs lint, format:check, typecheck, test (with coverage), build, and audit on every PR.
• Rationale: Partial gates are a false sense of safety; agents rely on the gate set being complete to validate their output.
• Evidence expected: Workflow inspection; gates enumerated.

DLM-020: Deployment pipeline documented

• Pillar: Delivery & Measurement
• Level: 2
• Scope: application
• Check type: deterministic
• Description: A staging → production promotion is documented, with an explicit rollback path, in docs/runbook.md or the CI config.
• Rationale: Agents making risky changes need to know how to unwind them quickly.
• Evidence expected: Documentation; release workflow with environment promotion.

DLM-030: Deploy previews on PRs

• Pillar: Delivery & Measurement
• Level: 2
• Scope: application
• Check type: deterministic
• Description: Pull requests produce an isolated preview URL (Vercel, Cloudflare Pages, Netlify previews, or equivalent).
• Rationale: Reviewing a live preview is the difference between catching a regression and shipping one.
• Evidence expected: Preview URL comments on recent PRs.

DLM-040: CI feedback time instrumented

• Pillar: Delivery & Measurement
• Level: 3
• Scope: repository
• Check type: manual
• Description: Median CI duration is tracked and kept below 10 minutes for typical PRs.
• Rationale: Slow CI compounds: agents can run fewer iterations per day, and humans context-switch.
• Evidence expected: Metrics dashboard or CI timing data over the last 30 days.

DLM-050: DORA metrics captured

• Pillar: Delivery & Measurement
• Level: 3
• Scope: organization
• Check type: manual
• Description: The four DORA metrics (deployment frequency, lead time for changes, change failure rate, mean time to restore) are measured and reviewed.
• Rationale: DORA is the industry-standard leading indicator of delivery health; agents should have access to the same signals humans do.
• Evidence expected: Dashboard or report; cadence of review.

DLM-060: Product analytics stubbed

• Pillar: Delivery & Measurement
• Level: 2
• Scope: application
• Check type: deterministic
• Description: A product-analytics client is stubbed in (lib/telemetry/analytics.ts or equivalent) — disabled by default but wired so that enabling it is a config flip.
• Rationale: Agents cannot reason about user impact without behavioral data; retrofitting analytics is painful.
• Evidence expected: Analytics module present; feature flag controls it.

DLM-070: Self-improving gates

• Pillar: Delivery & Measurement
• Level: 4
• Scope: organization
• Check type: manual
• Description: Orchestration exists that can auto-propose remediations for failing CARL criteria (e.g., an agent that opens a PR to add an AGENTS.md when DOC-030 fails).
• Rationale: Level 5 is the level at which the system measurably improves itself. Automation is both signal and mechanism.
• Evidence expected: Automation config; history of merged auto-remediation PRs.

DLM-080: Billing or payments integrated — a revenue path exists

• Pillar: Delivery & Measurement
• Level: 5
• Scope: application
• Check type: deterministic
• Description: A billing or payments integration (payment-provider SDK) is present.
• Rationale: A system an agent could turn into a revenue-capable business needs a wired path to charge customers; the payment integration is the load-bearing signal.
• Evidence expected: A payment-provider dependency (e.g. Stripe) detected in a manifest.

DLM-090: Operations runbooks present — autonomous ops are documented

• Pillar: Delivery & Measurement
• Level: 6
• Scope: repository
• Check type: deterministic
• Description: Operations runbooks (incident / on-call / scaling procedures) are present.
• Rationale: A business an agent runs needs executable operational knowledge; runbooks are the machine-readable procedures for keeping it alive without a human in the loop.
• Evidence expected: Runbook artifacts detected (e.g. RUNBOOK.md, docs/runbook.md, or **/runbooks/**).

Appendix A — Applicability

A criterion may be marked not applicable when it genuinely does not pertain — for example, BLD-070 (Docker image) for a library package never intended to run as a service. Applicability markings MUST be justified in the report and reviewable.

Appendix B — Variance for AI-assisted criteria

Criteria with check type: ai-assisted MUST disclose measured variance when published in a conformance report. See spec/v1.0/standard.md §6.2. Target variance: < 2%.

Appendix C — Contributing new criteria

New criteria proposals go through the standard lifecycle (working draft → candidate → ratified) documented in GOVERNANCE.md. Use the standard_change issue template to propose.

— End of Criteria Catalog v1.0.0 —

Related

• The Standard v1.0 — where these criteria come from.
• Assessment methodology — how an assessor evaluates each criterion.
• Criterion index — searchable table with one-line summaries and links back into this catalog.