> ## Documentation Index
> Fetch the complete documentation index at: https://docs.causeloop.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Research API

> The legacy research/clustering surface in services/api/main.py: why it isn't deployed, how to run it locally, and a grouped tour of its domains.

`services/api/main.py` is a second, separate FastAPI application — not a superset of `product_app.py`, and not imported by it. It hosts the historical research workbench that predates the tenant product: causal-graph schema exploration, dataset import and clustering experimentation, artifact/lineage inspection, the strict MVP1 250-issue analysis pipeline, and a file-backed client onboarding API that predates the real tenant-schema onboarding pipeline documented under [Client onboarding pipeline](/features/client-onboarding-pipeline). It is large — as of this writing its `FastAPI()` app registers 279 distinct paths (334 method+path operations), against the product API's 44.

<Warning>
  This page documents research/local-only surface area. Nothing here is reachable at `api.causeloop.ai`, is exercised by the tenant console, or should be treated as a stable contract — it changes freely as research work continues, with no allowlist and no `product_app.py`-style import-time enforcement holding it to a fixed shape.
</Warning>

## Why it isn't deployed

Three structural reasons, not just "it's old code":

1. **It's a different process boundary entirely.** The repo's `Dockerfile` starts `uvicorn services.api.product_app:app`; nothing in the deployed container ever imports `services.api.main`. `product_app.py`'s own module docstring states the reasoning directly: importing `main`'s app "creates stores and exposes operations the customer/staff product does not need."
2. **No tenant isolation.** Its routes operate on process-global, dataset-id-keyed state (`causegraph.storage.memory.CauseLoopState`) rather than the per-tenant, RLS-isolated Postgres schemas the product uses. The remediation router's own docstring calls out the specific example: the legacy `/mvp1/caps` / `/mvp1/reviews` system this app hosts "has no tenant isolation at all (dataset\_id-keyed, process-global)," which is exactly why `services/api/remediation_api.py` was built as a real, tenant-scoped replacement rather than extended in place.
3. **It does real work on request threads.** Parts of this surface (`services/api/clients_api.py`'s legacy onboarding flow, included here alongside the shared product routers) spawn `threading.Thread(..., daemon=True)` to run background work — a pattern the product API's routers never use, since a daemon thread's work is lost if the process restarts mid-request. See [System overview](/architecture/overview#no-daemon-thread-work-from-api-requests) for why that distinction matters operationally.

`main.py` does include the same tenant-facing routers the product uses (`auth_router`, `onboarding_router`, `sources_router`, `insights_router`, `members_router`, `remediation_router`, `metrics_router`) alongside its own — so in local `basic`/`mvp1`/`relational` modes you can reach both the tenant/staff product routes *and* the research surface on the same port, unlike production where only the allowlisted 44 operations exist at all.

## Exploring it locally

The research surface has three local modes, from lightest to heaviest (`docs/LOCAL_SETUP.md`); see [Research modes](/onboarding/research-modes) for the full walkthrough of each.

<Steps>
  <Step title="Install dependencies">
    ```bash theme={null}
    uv sync --dev
    npm ci --prefix apps/web
    uv run python scripts/check_local_setup.py --mode basic
    ```

    The doctor is local and non-mutating — it never calls OpenAI or a remote service.
  </Step>

  <Step title="Start basic mode">
    Needs neither Docker nor an API key:

    ```bash theme={null}
    uv run python scripts/run_local.py
    ```

    or two terminals:

    ```bash theme={null}
    uv run uvicorn services.api.main:app --host 127.0.0.1 --port 8000 --reload
    npm --prefix apps/web run dev -- --host 127.0.0.1
    ```
  </Step>

  <Step title="Open it">
    * Web UI (the Vite workbench, `apps/web`): `http://127.0.0.1:5173`
    * Swagger UI: `http://127.0.0.1:8000/docs`
    * Live schema: `http://127.0.0.1:8000/openapi.json`

    A fresh clone has the tracked synthetic banking dataset available in this mode, but no pre-generated strict MVP1 report — report-read endpoints correctly `404` until you run an analysis.
  </Step>
</Steps>

Two heavier modes build on `basic`, both opt-in and both requiring explicit user approval before any external call:

| Mode         | Adds                                               | Requires                                                                                                                                                                             |
| ------------ | -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `mvp1`       | The strict, governed 250-issue OpenAI analysis run | An OpenAI key in `.env.local`, and — inside the UI — two separate explicit transfer approvals before the run sends de-identified issue fields externally                             |
| `relational` | PostgreSQL-backed analysis reviewed through Excel  | Docker Compose Postgres (`infra/docker-compose.yml`), plus the same explicit-approval OpenAI transfer flow, run via `workers/run_banking_relational_e2e.py --approve-openai-llm ...` |

Neither of these is ever something a code agent should trigger without the user's explicit, in-the-moment approval of the OpenAI cost and data transfer — that approval can't be inferred from a task description. See [Research modes](/onboarding/research-modes) and [MVP1 pipeline](/research/mvp1-pipeline) for the full contract.

## A grouped tour of the surface

The routes below are organized by domain, not registration order, to make the \~280 paths navigable. None of these paths are on the product allowlist (see [API reference overview](/api-reference/overview#the-allowlist-philosophy)) — treat every one as local/research-only.

### Graph schema and registry

`GET /graph/schema` returns the full causal-graph schema (`causegraph.graph_schema.full_graph_schema()`) — node/edge type definitions for the provenance graph model this whole research pipeline is built around. `GET /lifecycle/graph` and the `/artifacts/registry`, `/artifacts/validation` family expose the artifact-kind registry and its internal-consistency checks. `GET /artifacts/models/*` covers the separate model-training-artifact registry (trainers, targets, a "model × data" coverage matrix, checkpoint registration/retirement) — distinct from the product's `model_checkpoints` table entirely; this is research-pipeline model bookkeeping.

### Datasets

`/datasets/import`, `/datasets/status`, `/datasets/public/sources`, `/datasets/narrative/adapters`, `/datasets/narrative/import`, `/datasets/imported`, `/datasets/{dataset_id}/summary` load and inspect datasets into the process-global `CauseLoopState` — including the built-in public dataset acquisition workers (`workers/ingest/acquire_public_datasets.py`) that seed the tracked synthetic banking dataset used by `basic` mode.

### Runs, artifacts, and lineage

`/artifacts/object-storage/*` is a large sub-family covering export/read/write, promotion between primary/secondary storage, and lifecycle validation for the research pipeline's own object-storage abstraction — a separate concern from the product's `services/storage/object_store.py`. `/artifacts/lineage` and `/artifacts/{artifact_id}/lineage` expose OpenLineage-style provenance for a given artifact; `/runs`, `/tasks/runs/recent`, `/tasks/runs/{artifact_id}` list and inspect individual pipeline executions. `/mvp1/runs`, `/mvp1/runs/{artifact_id}`, and `/mvp1/runs/{artifact_id}/lineage` are the equivalent read surface scoped specifically to strict MVP1 runs (see below).

### Themes and clustering

`/clusters/run` and `/clusters/algorithms` execute and enumerate the clustering algorithms available to the research pipeline — a materially larger set than the product's fixed `hdbscan` default (`TrainingConfigRequest.clusterer` in `onboarding_api.py`), including `kmeans`, `leiden`, `bm25`, `tfidf` and several hybrid combinations. `/themes`, `/themes/{theme_id}/analysis`, `/themes/{theme_id}/issues`, `/themes/{theme_id}/fishbone`, and `/themes/llm-summaries*` cover theme-level exploration and LLM-generated summaries/fishbone diagrams once a clustering run has produced themes. `/loops/*` is an earlier, process-global causal-loop remediation surface (algorithms, review, CAPA drafting) — the direct conceptual predecessor of the tenant-scoped [Remediation](/features/remediation) system, kept here for research continuity rather than removed outright.

### Strict MVP1 reports

A stable, explicitly-versioned sub-contract defined separately in `services/api/mvp1_contract.py` (`MVP1_ENDPOINTS`, all under `/mvp1/*`) rather than inline in `main.py`, specifically so its transport shape can't silently drift from its own analysis logic. Each entry is tagged `product` or `support` audience and a minimum permission (`read`/`review`/`operate`) — `product` routes are the ones the Vite workbench's browser client actually calls; `support` routes exist for imports, diagnostics, and stage-by-stage execution but aren't duplicated in that client. Highlights: `GET /mvp1/algorithms`, `GET /mvp1/datasets`, `POST /mvp1/issue-intelligence/run` (the governed run itself, gated on `external_embedding_transfer_approved`/`external_llm_transfer_approved` flags in its request body), `GET /mvp1/issue-intelligence/latest` and its `.../export.xlsx` sibling, `GET /mvp1/caps` / `POST /mvp1/caps` / `PATCH /mvp1/caps/{cap_id}` (the process-global CAP system referenced above), `GET /mvp1/reviews` / `POST /mvp1/reviews/{review_id}/decision`, `GET /mvp1/timeline`, and the public-evidence pair `GET /mvp1/public-evidence/latest` / `POST /mvp1/public-evidence/run`. Every `Mvp1Request` subclass sets `model_config = ConfigDict(extra="forbid")`, so an unexpected field is a `422` rather than a silently-ignored typo — useful when frontend and backend drift during active research work. See [MVP1 pipeline](/research/mvp1-pipeline) for the full analysis contract.

### File-based `/clients` onboarding API

`services/api/clients_api.py` (mounted into `main.py`, not a standalone module) is the client onboarding flow that predates the real tenant-schema pipeline in `onboarding_api.py`. Its state is a flat directory tree, not Postgres: `ClientAssetStore` persists one `manifest.json` per client under `data/clients/<client_id>/` (path overridable with `CAUSELOOP_CLIENTS_ROOT`) — no schema-per-tenant, no row-level security, no migrations. Its shape mirrors the real pipeline closely enough to be a useful reference: `POST /clients` (create), `GET /clients` / `GET /clients/{client_id}`, `POST /clients/{client_id}/sources` (register a connector) and `.../sources/{source_id}/activate`, `.../sources/{source_id}/upload`, `POST /clients/{client_id}/ingest`, `PUT /clients/{client_id}/training-config`, `POST /clients/{client_id}/train` (`202`) with `GET /clients/{client_id}/train/status`, `POST /clients/{client_id}/activate`, `POST /clients/{client_id}/go-live`, and `GET /clients/{client_id}/insights/{collection}`.

Its connector registry (`GET /connectors`, `causegraph.onboarding.connectors.CONNECTORS`) is the source of the documented `409 not_enabled` stub behavior: only `file_upload` is `enabled = True`; `postgres`, `s3_dump`, and `kafka_pubsub` are registered (visible in the schema, listable, schema-validated) but raise `ConnectorNotEnabledError` — surfaced as `409 {"detail": {"status": "not_enabled", "message": "..."}}` — the moment you try to activate one. See [Errors and conventions](/api-reference/errors-and-conventions#not-yet-enabled-connectors-a-real-409-example) for the exact response shape.

## Product code must not depend on this surface

The dependency direction is one-way and enforced structurally, not just by convention: `product_app.py` imports `auth_api`, `onboarding_api`, `insights_api`, `members_api`, `metrics_api`, `remediation_api`, and `sources_api` — never `services.api.main`, never `clients_api`, never `mvp1_contract`, and never `causegraph.storage.memory.CauseLoopState`. If you're working on a product feature and find yourself reaching for something defined only in `main.py` or `clients_api.py`, that's a signal the product-side equivalent either already exists elsewhere (check `onboarding_api.py`, `remediation_api.py`, `sources_api.py` first) or genuinely needs to be built there — not imported from the research app.

## Related

* [API reference overview](/api-reference/overview)
* [Errors and conventions](/api-reference/errors-and-conventions)
* [Research modes](/onboarding/research-modes)
* [MVP1 pipeline](/research/mvp1-pipeline)