Prefix: TST- What it measures: whether the test suite gives agents a
reliable signal they can use to verify their own changes — unit, integration,
and end-to-end coverage; enforced thresholds; selector durability; isolation;
and flake detection.
Without tests, an agent either assumes its work is correct (it often isn't) or re-reads the diff forever. A fast, comprehensive, low-flake test suite is the single biggest leverage point for agent effectiveness at L2 and above.
Criteria in this pillar
TST-010 — Test framework present
- Level: 1 · Scope: application · Check: deterministic
- A test framework is configured (Vitest, Jest, pytest, Go test, etc.) and at least one test exists and passes.
- Rationale: agents can only verify their own changes if tests exist to run.
- Evidence expected: test runner config;
package.json.scripts.test; at least one green test.
TST-020 — Coverage measurement configured
- Level: 2 · Scope: application · Check: deterministic
- Coverage tooling runs in CI and emits text/html/lcov output. A coverage
command exists (
pnpm test:coverage). - Rationale: coverage is a crude but essential signal for agents choosing where to add tests.
- Evidence expected: Vitest
coverageconfig or equivalent; coverage artifact in CI output.
TST-030 — Coverage thresholds enforced at ≥ 80%
- Level: 3 · Scope: application · Check: deterministic
- CI fails if coverage drops below 80% on branches, functions, lines, and statements.
- Rationale: without enforcement, coverage metrics drift downward as features are added without tests.
- Evidence expected:
vitest.config.*or equivalent withthresholds: { branches: 80, functions: 80, lines: 80, statements: 80 }.
TST-040 — End-to-end tests for critical journeys
- Level: 3 · Scope: application · Check: deterministic
- At least one Playwright (or equivalent) E2E test exists covering signup/login (if applicable) and the core feature happy path.
- Rationale: agents cannot simulate browser interactions themselves; E2E coverage provides a verification layer they depend on.
- Evidence expected:
playwright.config.*+ at least one*.spec.tsundere2e/ortests/e2e/.
TST-050 — Resilient selectors in UI tests
- Level: 3 · Scope: application · Check: ai-assisted
- UI tests use
data-testid,role, or accessible label selectors rather than CSS classes or tag chains. - Rationale: CSS-selector-based tests break on styling refactors, which agents perform frequently. Semantic selectors remain stable.
- Evidence expected: grep on test files for
className-style selectors vsgetByRole/getByTestId; AI-assisted judgment on ratio.
TST-060 — Test isolation (no shared mutable state)
- Level: 4 · Scope: application · Check: ai-assisted
- Tests do not depend on execution order or shared database state. DB tests use transactions or per-test schemas.
- Rationale: flaky tests poison the signal agents rely on to verify their changes.
- Evidence expected: evidence of transaction setup / teardown; AI-assisted judgment; flake rate metric if collected.
TST-070 — Flake detection and quarantine
- Level: 4 · Scope: application · Check: deterministic
- CI detects flaky tests (via retry + failure-rate measurement) and tracks them in an issue or dashboard.
- Rationale: flakes are the silent killer of agent-driven delivery. Detection is the prerequisite for remediation.
- Evidence expected: configured retry + reporter; issue tracker or dashboard references.
TST-080 — Property-based or fuzz testing for critical modules
- Level: 5 · Scope: application · Check: manual
- Critical parsers, validators, or serializers have property-based
(
fast-check,hypothesis) or fuzz tests. - Rationale: example-based tests miss the adversarial inputs agents must handle. PBT raises the floor.
- Evidence expected: import of
fast-check/hypothesis/go-fuzz; at least one property test.
Related
- The 8 pillars — sibling pillars in v1.0.
- Criteria catalog — all criteria, one page.
- Maturity levels — how criteria combine into a level.