Testing and verification

Causeloop has two independent CI pipelines — backend/.github/workflows/product-ci.yml and frontend/.github/workflows/product-ci.yml — plus a set of local-only checks that are faster to run while iterating. This page is the canonical list of both. Run the checks that match what you changed; run the full backend and frontend CI sequence before handing work to another person or agent.

None of this is enforced by a make target. AGENTS.md is explicit that no Make command is required or expected — use the direct uv, npm, Docker Compose, and Alembic commands below.

Backend: Python

uv run ruff check .
uv run pytest

ruff targets Python 3.12 with a 100-character line length (pyproject.toml, [tool.ruff]), applied repo-wide. pytest picks up everything under backend/tests/ — over fifty modules covering the tenancy/RLS control plane, auth flows, onboarding, sources, remediation, the durable job worker, and both historical MVP1 pipelines (test_mvp1_*.py, test_banking_relational_runner.py, test_homecare_relational_*.py). test_local_setup_tools.py covers scripts/check_local_setup.py itself. If a sandbox redirects user caches, prefix uv commands with UV_CACHE_DIR=/tmp/uv-cache.

What backend CI actually runs

product-ci.yml’s backend job, in order:

Install and lint

uv sync --frozen --dev
uv run ruff check .

Unit and contract tests

uv run pytest -q

This step runs with in-process/local provider env vars set (CAUSELOOP_OBJECT_PROVIDER=memory, CAUSELOOP_RUNTIME_BACKEND=local, CAUSELOOP_REPOSITORY_BACKEND=local, CAUSELOOP_GRAPH_BACKEND=local, OPENAI_API_KEY="", among others) — no external services, no live LLM calls.

Build the backend image and prove it has no Neo4j driver

docker build --tag causeloop-backend:ci .
docker run --rm causeloop-backend:ci python -c \
  'import importlib.util; assert importlib.util.find_spec("neo4j") is None'

This guards a real regression class: the product image must not accidentally pull in the legacy graph-database dependency tree.

Validate both Compose files without starting containers

docker compose -f infra/docker-compose.yml config --quiet
docker compose -f infra/docker-compose.product.yml config --quiet

Start a production-parity local data plane

docker compose -f infra/docker-compose.product.yml up -d --build \
  mailpit postgres redis minio minio-init migrate local-admin \
  product-worker provisioner-worker api

Note this omits the frontend service — the backend job never builds or runs the Next.js console.

Wait for private API readiness

curl --fail --silent --show-error \
  --retry 90 --retry-delay 2 --retry-all-errors \
  --connect-timeout 2 --max-time 180 \
  http://127.0.0.1:18000/health/ready

/health/ready (implemented in services/api/product_app.py) only returns 200 once the database, both worker heartbeats, object storage, credential encryption, email delivery, and (when required) Redis are all ready — see Troubleshooting for what each of those checks means.

Live product integration tests

uv run pytest -q \
  tests/test_auth_flows.py \
  tests/test_durable_jobs.py \
  tests/test_metrics_api.py \
  tests/test_onboarding_api.py \
  tests/test_pipeline_materialize.py \
  tests/test_remediation_api.py \
  tests/test_sources_api.py \
  tests/test_tenancy_isolation.py

Run against the real Postgres/Redis/MinIO/Mailpit containers on their host-mapped ports (55433, 16379, 19000, 18025), not mocks.

Rehearse backup and restore

scripts/rehearse_product_backup.sh

Tear down

docker compose -f infra/docker-compose.product.yml down -v

CI intentionally uses -v to discard the ephemeral volumes; do not add -v to your own local down unless you deliberately intend to delete local product data (see Local product stack).

A separate terraform job in the same workflow runs terraform fmt -check/init -backend=false/validate against deploy/aws and compiles deploy/azure/main.bicepparam with the Azure CLI — infrastructure-as-code changes under deploy/ must pass that job too.

Backend workbench: `apps/web` (legacy research UI)

The Vite workbench used by the basic/mvp1 research modes (Research modes) has its own, much smaller check surface:

npm --prefix apps/web run test
npm --prefix apps/web run build

apps/web’s test script is tsc -b --noEmit — a type-check, not a runtime test suite. build is tsc -b && vite build. These are exactly the two web checks AGENTS.md calls out. The Makefile’s test-web target only runs npm run test (the type-check); the build is a separate command not covered by any Make target (make test runs test-python and test-web).

Frontend: Next.js console

npm ci
npm audit --audit-level=high
npm run audit:mvp:strict
npm run typecheck
npm test
npm run build

That is the exact sequence frontend/.github/workflows/product-ci.yml runs, followed by docker build --tag causeloop-frontend:ci .. A few of these are worth explaining:

npm test runs vitest run (see frontend/package.json). npm run test:watch runs the interactive vitest watcher for local iteration.
npm run typecheck is tsc --noEmit; npm run lint is next lint. Both are separate from npm test and are not bundled into it.
npm run audit:mvp / npm run audit:mvp:strict runs scripts/audit-mvp-surface.mjs, a TypeScript-AST scanner over src/. It walks every fetch/apiGet/apiSend/adminSend/tenantSend call, resolves the literal endpoint (including simple template-literal and constant-alias forms), and classifies it against a fixed family list: keep families like workspace-directory, authentication, staff-onboarding, staff-observability, and tenant-insights; target families the frontend must call (tenant-sources, tenant-members, tenant-remediation); and replace families flagged as legacy/compatibility — compatibility-session (/api/session), legacy-clients (/clients), legacy-connectors (/connectors), legacy-mvp1 (/mvp1), legacy-loops (/loops), and legacy-datasets (/datasets/status). Any call the scanner can’t classify at all fails the build even under plain npm run audit:mvp — that check isn’t strict-only. --strict additionally fails the build if any replace-family call site is present, if a required target family has zero call sites, or if production code imports the synthetic fixture, the retired report adapter, or the provenance module. CI always runs the :strict variant.
npm run e2e runs Playwright (playwright.config.ts), against http://localhost:3000 with a single Chromium project at a 1440×900 viewport, fullyParallel: false, and workers: 1. Its webServer block starts (or reuses) npm run dev for you, but the FastAPI product app must already be listening on http://127.0.0.1:18000 — start it with the product Compose stack first (see Local product stack). Playwright is not part of either CI workflow — it is a local/manual verification step. The unauthenticated redirect spec runs with no configuration; the authenticated specs in frontend/e2e/ (auth.spec.ts, console.spec.ts) require a seeded tenant and are explicitly skipped unless all three of these are set:

CAUSELOOP_E2E_WORKSPACE=tenant-slug \
CAUSELOOP_E2E_EMAIL=user@example.com \
CAUSELOOP_E2E_PASSWORD='...' \
npm run e2e

The old fake-session, /mvp1-mutation, and frozen-demo screenshot Playwright harnesses were deliberately removed from this repo — they no longer exercised the real product contract. Do not resurrect that pattern; write new specs against the seeded-tenant flow above.

Product-stack smoke test

python3 scripts/check_local_setup.py --mode product
docker compose -f infra/docker-compose.product.yml config
python3 scripts/smoke_product.py

scripts/smoke_product.py is a provider-neutral release check: it hits GET /health/live and GET /health/ready through the frontend’s /api/backend proxy (or a direct --api-url you pass), asserting {"status": "ok"} and {"status": "ready", ...} respectively, then confirms GET /login returns an HTML document. It defaults to http://127.0.0.1:13000 — the Compose frontend port — and is meant to be run against a stack that is already up, unlike the CI job above which builds and starts one from scratch. docker compose ... config (with either compose file) validates configuration only — interpolates env vars, resolves anchors, and checks YAML/schema validity — without starting any container. Run it after touching either compose file.

Before handing off work

Run the checks appropriate to the files you changed, at minimum:

uv run ruff check .
uv run pytest
python3 scripts/check_local_setup.py --mode product
docker compose -f infra/docker-compose.product.yml config
python3 scripts/smoke_product.py
docker compose -f infra/docker-compose.yml config
npm --prefix apps/web run test
npm --prefix apps/web run build
npm run audit:mvp:strict   # from frontend/
npm run typecheck          # from frontend/
npm test                   # from frontend/
npm run build              # from frontend/

If you touched only the frontend or only backend Python, you can skip the other side’s suite — but if you changed a shared contract (a product_app.py route, a src/lib/api/*.ts client function, or anything in the /admin/tenants/*, /insights, /sources, /members, or /remediation/* families), run both, because audit:mvp:strict and the backend’s product contract test (tests/test_product_app_contract.py) each independently enforce the same allowlist from their own side.

Welcome

Developer Onboarding

Testing and verification

Backend: Python

What backend CI actually runs

Backend workbench: `apps/web` (legacy research UI)

Frontend: Next.js console

Product-stack smoke test

Before handing off work

​Backend: Python

​What backend CI actually runs

​Backend workbench: apps/web (legacy research UI)

​Frontend: Next.js console

​Product-stack smoke test

​Before handing off work

​Related

Backend: Python

What backend CI actually runs

Backend workbench: `apps/web` (legacy research UI)

Frontend: Next.js console

Product-stack smoke test

Before handing off work

Related