The CARL CLI is designed to be CI-native. Its exit codes are stable, its JSON output is a stable contract, and it runs in under a minute on most repositories. This guide walks through three common CI integrations:
- Running CARL on every PR and failing the build if the level drops.
- Posting the report as a PR comment.
- Publishing the JSON report as a workflow artifact for downstream tools.
The examples use GitHub Actions; the same approach applies to GitLab CI, CircleCI, Buildkite, and every other modern CI.
Prerequisites
- A repository that already runs CI on every PR.
- Node 22+ available on CI runners (GitHub Actions
ubuntu-latesthas this by default once you setactions/setup-node@v4). pnpmornpm(both work).
Step 1 — Add a CARL step to the workflow
# .github/workflows/ci.yml
name: CI
on:
pull_request:
branches: [main]
push:
branches: [main]
jobs:
carl:
name: CARL assessment
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write # for the comment step below; safe to omit if unused
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 1
- uses: pnpm/action-setup@v4
with:
version: 9
run_install: false
- uses: actions/setup-node@v4
with:
node-version-file: .nvmrc
cache: pnpm
- run: pnpm install --frozen-lockfile
- name: Run CARL
run:
pnpm dlx @wentzel/carl assess --min-level 3 --json > carl-report.json
JSON output goes to stdout; the CLI's progress line only appears for the pretty
format, so no extra flag is needed in CI. The --min-level 3 flag makes the CLI
exit non-zero if the assessed level drops below L3 (Reviewed). --min-level
accepts any rung on the 0–6 ladder (0 Manual, 1 Scripted, 2 Tested, 3 Reviewed,
4 Agent-driven, 5 Creates a business, 6 Runs a business). Tune the threshold to
your team's current level — raising it later is straightforward.
Step 2 — Upload the report as an artifact
- name: Upload CARL report
if: always() # upload even if the min-level gate failed
uses: actions/upload-artifact@v4
with:
name: carl-report
path: carl-report.json
retention-days: 30
The if: always() is important — when the level gate fails, you still want the
artifact to debug why.
Step 3 — Comment the summary on the PR
-
Add a small script to render the report JSON as Markdown:
// scripts/ci/carl-pr-comment.mjs import fs from 'node:fs/promises'; const report = JSON.parse(await fs.readFile('carl-report.json', 'utf8')); const lines = [ `### CARL Assessment — L${report.levelAchieved}`, '', '| Pillar | Score |', '|---|---|', ...Object.entries(report.pillarScores).map( ([k, v]) => `| ${k} | ${(v * 100).toFixed(0)}% |`, ), ]; const top = report.findings .filter((f) => f.result === 'fail' && f.remediationText) .slice(0, 3); if (top.length > 0) { lines.push('', '#### Top remediations'); for (const f of top) { lines.push( `- **${f.criterionId}** (${f.severity}) — ${f.remediationText}`, ); } } await fs.writeFile('carl-pr-comment.md', lines.join('\n')); -
Post the comment:
- name: Render PR comment if: github.event_name == 'pull_request' run: node scripts/ci/carl-pr-comment.mjs - name: Post PR comment if: github.event_name == 'pull_request' uses: marocchino/sticky-pull-request-comment@v2 with: path: carl-pr-comment.md header: carl-assessmentThe
sticky-pull-request-commentaction replaces the prior bot comment on each push, so the PR never accumulates stale assessments.
Step 4 — Require the check in branch protection
In GitHub → Settings → Branches → default-branch rule, add CARL assessment to
the list of required status checks. With the rule in place, a PR that drops the
level below --min-level cannot be merged without an admin override.
Strategies for introducing CARL to an existing repository
-
First run, no gating. Remove
--min-level Nand just publish the report as an artifact. Let the team see the current level before anything gates on it. -
Ratchet up. Once the team agrees on the current level, set
--min-levelto it and protect the branch. The CI gate will now prevent accidental regression. -
Announce increments. When the team clears the next level, announce it in a PR, bump
--min-level, and update the badge. Use the remediation list in the report to plan the next target.
Monorepos
For monorepos, use the same workflow but point at application directories:
- name: CARL — web app
run:
pnpm dlx @wentzel/carl assess ./apps/web --min-level 3 --json >
web-report.json
- name: CARL — engine
run:
pnpm dlx @wentzel/carl assess ./packages/engine --min-level 3 --json >
engine-report.json
Or run once at the repo root and let the CLI enumerate applications — see Monorepo setup for details.
Caching
The CARL CLI is fast enough that caching its output usually doesn't pay. If your repository is genuinely large (tens of thousands of files) and your CI feedback time matters, consider:
- Caching the CARL install by using
pnpm installinstead ofpnpm dlx. - Running CARL only on PRs that touch source files (use
paths-ignoreto exclude docs-only PRs).
Related
- CLI — the command surface CARL exposes.
- Reading your report — interpreting what CI surfaces on the PR.
- Monorepo setup — how CARL handles per-application assessment in monorepos.