> ## Documentation Index
> Fetch the complete documentation index at: https://docs.causeloop.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Research API surface
> A domain-by-domain tour of the research application's ~279-path surface: schema, datasets, embeddings, runs and lineage, clustering, process mining, strict MVP1 reports, and the file-based clients API.
`services/api/main.py` registers 279 distinct paths — 334 method+path operations (per `app.openapi()`). That number is too large to read route-by-route, so this page groups it by domain instead of registration order. For *why* this surface exists and isn't deployed, see [Research workbench overview](/research/overview) and [Research API](/api-reference/research-api); this page is the detailed reference for *what's in it*.
Every route below lives only in `services/api/main.py` (or a router it mounts, like `services/api/clients_api.py`). None of it is reachable from `api.causeloop.ai`, none of it is on the product allowlist described in [API reference overview](/api-reference/overview#the-allowlist-philosophy), and its shape can change without the deprecation process a real product contract would get.
## Explore it locally
The fastest way to see the live, authoritative shape of this surface — more reliable than any document, including this one — is to run it and open its interactive schema:
```bash theme={null}
uv run python scripts/run_local.py
```
Then open:
* Swagger UI: `http://127.0.0.1:8000/docs`
* Machine-readable schema: `http://127.0.0.1:8000/openapi.json`
This is `basic` mode, the lightest of the three research modes — no Docker, no OpenAI key. See [Research modes](/onboarding/research-modes) for `mvp1` and `relational`, which add a real analysis run on top of the same API.
## Graph schema and registry
The provenance graph model this entire research pipeline is built around, plus the internal-consistency checks that keep artifact bookkeeping honest.
| Endpoint | Purpose |
| -------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `GET /graph/schema` | Full causal-graph schema (`causegraph.graph_schema.full_graph_schema()`) — node/edge type definitions, 31 node types and 34 edge types as of this writing, spanning nine of the ten defined graph layers (`technical_lineage` is reserved but unused) |
| `GET /lifecycle/graph` | A bounded slice of the lifecycle graph for a limited number of cases, for UI rendering |
| `GET /artifacts/registry`, `GET /artifacts/validation` | The artifact-kind registry and its internal-consistency validation |
| `GET /artifacts/models`, `GET /artifacts/models/model-data-matrix`, `POST /artifacts/models/train` | A separate model-training-artifact registry (trainers, targets, a model-by-data coverage matrix, checkpoint registration) — distinct from the product's `model_checkpoints` table; this is research-pipeline bookkeeping only |
## Datasets and records
Loads and inspects datasets into the process-global `CauseLoopState`, including the public-dataset acquisition program.
| Endpoint | Purpose |
| --------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `POST /datasets/import` | Import a dataset (`helpdesk` or `bpic13`) from a local file path, or fall back to small built-in demo records via `allow_demo_data` |
| `GET /datasets/status` | Loaded record counts per dataset, narrative-record counts, and — for `helpdesk`/`bpic13` specifically — whether the real source file (`data/raw/helpdesk/finale.csv`, `data/raw/bpic13/BPI_Challenge_2013_incidents.xes.gz`) exists on disk |
| `GET /datasets/public/sources` | Full public-source catalog with per-source lifecycle status: downloaded, canonicalized, imported, indexed, embedded, included in the active theme — see [Datasets](/research/datasets) for what this actually reports today |
| `GET /datasets/narrative/adapters`, `POST /datasets/narrative/import` | Narrative-adapter readiness and import for the `Narrative`/`EvidenceChunk`/`ExtractedObservation` normalized shape public sources feed into |
| `GET /datasets/{dataset_id}/summary` | Per-dataset record counts and readiness for one dataset ID |
| `GET /records/{record_id}/provenance` | Provenance graph for a single record, independent of which case it belongs to |
## Embeddings and indexes
| Endpoint | Purpose |
| ------------------------------------ | --------------------------------------------------------------------------------- |
| `POST /embeddings/run` | Run the embedding provider over pending records |
| `POST /indexes/bm25/run` | Rebuild the BM25 lexical index over currently loaded documents |
| `GET /index/status`, `GET /overview` | Combined index/embedding/graph-schema overview used to drive workbench dashboards |
| `GET /providers/embeddings/status` | Health check for the configured embedding provider |
## Runs, artifacts, and lineage
| Endpoint | Purpose |
| ---------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `GET /runs`, `GET /tasks/runs/recent`, `GET /tasks/runs/{artifact_id}` | List and inspect individual pipeline executions and their task-run artifacts |
| `GET /artifacts/lineage`, `GET /artifacts/{artifact_id}/lineage` | OpenLineage-style provenance for a specific artifact |
| `GET /artifacts/object-storage/status`, `GET /artifacts/object-storage/export`, `POST /artifacts/object-storage/write` | Export/read/write and promotion between primary/secondary storage for the research pipeline's own object-storage abstraction — a separate concern from the product's `services/storage/object_store.py` |
| `GET /mvp1/runs`, `GET /mvp1/runs/{artifact_id}`, `GET /mvp1/runs/{artifact_id}/lineage` | The equivalent read surface scoped specifically to strict MVP1 runs (see below) |
## Themes, clustering, and review
Clustering execution, theme interpretation, and the process-global predecessor of the tenant remediation system.
| Endpoint | Purpose |
| ------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `POST /clusters/run`, `GET /clusters/algorithms` | Execute and enumerate clustering algorithms — `kmeans`, `leiden`, `bm25`, `tfidf`, and hybrid combinations, a materially larger set than the product's fixed `hdbscan` default (`TrainingConfigRequest.clusterer` in `onboarding_api.py`) |
| `GET /themes`, `GET /themes/{theme_id}/analysis`, `GET /themes/{theme_id}/issues`, `GET /themes/{theme_id}/fishbone` | Theme-level exploration once a clustering run has produced themes |
| `POST /themes/llm-summaries`, `GET /themes/llm-summaries/latest` | LLM-generated theme summaries |
| `GET /loops`, `GET /loops/active`, `POST /loops/run`, `POST /loops/{loop_id}/review`, `POST /loops/{loop_id}/capa/draft` | An earlier, process-global causal-loop remediation surface — algorithms, review, CAPA drafting — the direct conceptual predecessor of the tenant-scoped [Remediation](/features/remediation) system, kept here for research continuity rather than removed outright |
| `POST /reviews` | A standalone review-request recorder, independent of both `/loops/{loop_id}/review` and the MVP1 review queue below |
## Process discovery and conformance
`causegraph.tasks` (`TASK_SPECS`) is a 16-task catalog spanning process mining, issue intelligence, safety/cyber/reliability classification, and more; this section tours its event-log/process-mining family — process discovery, conformance/deviation, predictive process monitoring, process anomaly detection, and process-stub stitching. Each of those five tasks lists an implemented baseline plus named future providers that are cataloged but not wired (status `adapter_ready`) — Split Miner, alignment-based checking, ProcessTransformer, and graph autoencoders, for example. The other 11 tasks in the catalog (e.g. issue classification, narrative failure extraction, CAPA recommendation, maintenance RUL, cyber prioritization) target datasets such as OSHA, MAUDE, NASA C-MAPSS, and VCDB; six of them have no implemented baseline yet, only `adapter_ready` provider slots.
| Endpoint | Purpose |
| -------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `GET /tasks` | List all 16 catalog entries: each task's product function, description, target datasets, and its algorithms with implementation status — not just the five process-mining tasks toured on this page |
| `POST /tasks/{task_id}/run` | Execute a task's implemented baseline algorithm against imported event-log data |
| `GET /workflows/patterns`, `GET /workflows/{workflow_id}/graph` | Directly-follows workflow patterns and per-workflow graphs derived from case traces |
| `GET /cases`, `GET /cases/{case_id}`, `GET /cases/{case_id}/provenance` | List cases, inspect one case's event trace, and its provenance graph |
| `GET /trace/{node_type}/{node_id}` | Generic node-level trace lookup across the graph, independent of case or record framing |
| `GET /benchmarks/latest`, `POST /benchmarks/run`, `GET /benchmarks/tasks/latest`, `POST /benchmarks/tasks/run` | Benchmark runs against the task catalog above, for tracking baseline-vs-future-provider quality over time |
## Strict MVP1 reports
A stable, explicitly-versioned sub-contract defined separately in `services/api/mvp1_contract.py` (`MVP1_ENDPOINTS`, all under `/mvp1/*`) rather than inline in `main.py`, specifically so its transport shape can't silently drift from its own analysis logic. Every entry there is tagged `product` or `support` audience and a minimum permission (`read`/`review`/`operate`); `product` routes are the ones the Vite workbench's browser client actually calls. See [MVP1 pipeline](/research/mvp1-pipeline) for the analysis contract itself.
| Endpoint | Purpose |
| ---------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `GET /mvp1/algorithms`, `GET /mvp1/datasets` | List strategies and datasets available to the strict pipeline |
| `POST /mvp1/pipeline/run` | Run or resume the governed strict pipeline — gated on `external_embedding_transfer_approved` and `external_llm_transfer_approved` request-body flags, both required and both independent |
| `GET /mvp1/issue-intelligence/latest`, `GET /mvp1/issue-intelligence/latest/export.xlsx` | Read the latest completed strict artifact without recomputing, and its interactive-report XLSX export |
| `GET /mvp1/caps`, `POST /mvp1/caps`, `PATCH /mvp1/caps/{cap_id}` | The process-global CAP system referenced above — no tenant isolation, dataset-id-keyed |
| `GET /mvp1/reviews`, `POST /mvp1/reviews/{review_id}/decision` | The MVP1 review queue and optimistic-concurrency review decisions |
| `GET /mvp1/timeline` | Source and CauseLoop lifecycle events for an issue, cluster, or CAP |
| `GET /mvp1/public-evidence/latest`, `POST /mvp1/public-evidence/run` | Plan or retrieve public supporting evidence for specific issues |
Every `Mvp1Request` subclass sets `model_config = ConfigDict(extra="forbid")`, so an unexpected field is a `422` rather than a silently-ignored typo — useful when frontend and backend drift during active research work. The full contract, including the operator/developer-only support routes intentionally absent from the browser client, is in `docs/MVP1_API_CONTRACT.md`.
## File-based `/clients` onboarding API
`services/api/clients_api.py` (mounted into `main.py`, not a standalone module) is the client onboarding flow that predates the real tenant-schema pipeline in `onboarding_api.py`. Its state is a flat directory tree, not Postgres: `ClientAssetStore` persists one `manifest.json` per client under `data/clients//` (path overridable with `CAUSELOOP_CLIENTS_ROOT`) — no schema-per-tenant, no row-level security, no migrations.
| Endpoint | Purpose |
| ----------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `POST /clients`, `GET /clients`, `GET /clients/{client_id}` | Create and list clients, backed by one JSON manifest file per client |
| `POST /clients/{client_id}/sources`, `POST /clients/{client_id}/sources/{source_id}/activate`, `POST /clients/{client_id}/sources/{source_id}/upload` | Register a connector, activate it, and upload a file for `file_upload`-kind sources |
| `POST /clients/{client_id}/ingest`, `PUT /clients/{client_id}/training-config` | Normalize an uploaded file into a dataset version and configure training/LLM settings |
| `POST /clients/{client_id}/train`, `GET /clients/{client_id}/train/status` | Kick off training as a background daemon thread and poll its status — the pattern flagged in [Research workbench overview](/research/overview#the-hard-boundary) as one the product API's routers deliberately never use |
| `POST /clients/{client_id}/activate`, `POST /clients/{client_id}/go-live` | Activate a trained checkpoint and mark a client live |
| `GET /clients/{client_id}/insights/{collection}` | Read a computed model- or LLM-backed insight collection, `202`-polling while an LLM collection generates in the background |
Its connector registry (`GET /connectors`, `causegraph.onboarding.connectors.CONNECTORS`) is the source of the documented `409 not_enabled` stub behavior: only `file_upload` is `enabled = True`; `postgres`, `s3_dump`, and `kafka_pubsub` are registered — visible in the schema, listable, schema-validated — but raise `ConnectorNotEnabledError`, surfaced as `409 {"detail": {"status": "not_enabled", "message": "..."}}`, the moment you try to activate one. See [Errors and conventions](/api-reference/errors-and-conventions#not-yet-enabled-connectors-a-real-409-example) for the exact response shape.
## What this page leaves out
`main.py` also registers a large `/runtime/production-promotion/*` family (evidence bundles, checklists, bootstrap scripts) and a `/product/*` family that shadows some product-router names — these are operational tooling for the platform's own promotion process, not part of the research analysis surface this page tours, and are out of scope here. Use `/docs` locally if you need to look one up.
## Related
* [Research workbench overview](/research/overview)
* [MVP1 pipeline](/research/mvp1-pipeline)
* [Datasets](/research/datasets)
* [Research API](/api-reference/research-api)