> ## Documentation Index
> Fetch the complete documentation index at: https://docs.causeloop.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Research workbench overview

> What the pre-product research platform is, why it is preserved next to the product, and the structural boundary that keeps it from leaking into deployed code.

Causeloop ships two FastAPI applications from one repository. `services/api/product_app.py` is the deployed tenant/staff product — 44 allowlisted operations, described in [API reference overview](/api-reference/overview). `services/api/main.py` is everything that came before it: a much larger, process-global research application that predates tenancy, row-level security, and the allowlist model entirely. This section of the docs (`/research/*`) is the canonical reference for that second application — what it is, why it still lives in the repository, and the one rule that keeps it from becoming a liability: **product code must not import it.**

<Warning>
  Nothing under `/research/*` describes deployed behavior. `services/api/main.py` and its browser client, `apps/web`, never run in production — no Dockerfile, Render service, or Vercel deployment starts either of them. Treat this section as internal platform documentation for engineers doing research or pipeline-development work locally, not as a contract anyone outside the team depends on.
</Warning>

## What the research workbench is

`services/api/main.py` and the `causegraph` package underneath it (`packages/causegraph/src/causegraph/`) are a single experimentation platform built to answer three questions before the tenant product existed: what does a causal-issue graph model look like, how do you cluster and interpret large volumes of unstructured issue text, and what can process-mining techniques (event logs, directly-follows graphs, conformance checking) tell you about the workflows that produce those issues. Three components make up the surface:

1. **A causal-graph schema and process-global store.** `causegraph.graph_schema.full_graph_schema()` defines the full provenance graph model — as of this writing, 31 node types and 34 edge types across nine populated layers (`source_evidence`, `observation_normalization`, `process_workflow`, `issue_failure`, `remediation_outcome`, `knowledge_taxonomy`, `shared_pattern_anonymization`, `provenance_governance`, `data_science_rag_ml`); a tenth layer, `technical_lineage`, is defined in the `GraphLayer` enum but not yet used by any node or edge type. `GET /graph/schema` serves it directly. Every research route operates against `causegraph.storage.memory.CauseLoopState`, an in-process, dataset-id-keyed store — not the tenant-scoped, row-level-secured PostgreSQL schemas the product uses. See [Datasets](/research/datasets) for what gets loaded into that store.
2. **Clustering and theme interpretation.** `/clusters/run` and `/clusters/algorithms` expose a materially larger algorithm set than the product's fixed `hdbscan` default — `kmeans`, `leiden`, `bm25`, `tfidf`, and hybrid combinations — plus `/themes/*` for theme-level exploration, LLM-generated summaries, and fishbone diagrams once a run has produced clusters. The strict, governed variant of this same idea — an audited 250-issue severity, clustering, and root-cause pipeline — is documented separately in [MVP1 pipeline](/research/mvp1-pipeline).
3. **Process discovery and conformance.** `causegraph.tasks` defines a 16-task catalog (`TASK_SPECS`) spanning process mining, issue intelligence, and safety/cyber/reliability classification. Five of those tasks — process discovery, conformance/deviation, predictive process monitoring, process anomaly detection, and process-stub stitching — are the event-log/process-mining family and each has an implemented baseline algorithm (directly-follows graphs, PM4Py Inductive Miner, dominant-path deviation, a Markov next-activity baseline, overlap-graph stitching) plus named future providers (Split Miner, alignment-based checking, ProcessTransformer, graph autoencoders) that are cataloged but not wired. The remaining 11 tasks target other datasets (OSHA, MAUDE, NASA C-MAPSS, VCDB, and more); six of them have no implemented baseline yet, only `adapter_ready` provider slots. `GET /tasks` lists all 16 catalog entries; `POST /tasks/{task_id}/run` executes a task's implemented baseline against imported data (BPIC13, Helpdesk, and other adapters, depending on the task). See [API surface](/research/api-surface) for the full grouped tour, including the file-based `/clients` onboarding API that predates the real tenant pipeline.

```mermaid theme={null}
flowchart TB
    subgraph Research workbench — never deployed
        CG["causegraph package<br/>graph schema, clustering,<br/>process discovery"]
        M["services/api/main.py<br/>279 paths / 334 operations"]
        W["apps/web<br/>Vite workbench :5173"]
        CG --> M --> W
    end
    subgraph Product boundary — deployed
        PA["services/api/product_app.py<br/>44 allowlisted operations"]
        F["frontend console<br/>Next.js"]
        PA --> F
    end
    M -. "explicitly not imported by" .-> PA
```

## Why it is preserved

This is not dead code kept out of inertia. Three concrete reasons keep it in the repository:

* **It is the historical basis for real product systems.** The product's remediation feature (`services/api/remediation_api.py`) is a direct, tenant-scoped successor to the process-global `/loops/*` and legacy `/mvp1/caps` / `/mvp1/reviews` systems still hosted here; the product's onboarding pipeline (`services/api/onboarding_api.py`) is the successor to the file-based `/clients` API in `services/api/clients_api.py`. Keeping the predecessor in place gives engineers working on the current feature a working reference implementation to compare against.
* **It hosts research directions the product hasn't absorbed yet.** Process discovery, conformance checking, predictive process monitoring, and the wider public-dataset acquisition program (CISA KEV, NVD, OSHA, MAUDE, NHTSA, and more — see [Datasets](/research/datasets)) are active research areas with no tenant-product equivalent. Removing the workbench would delete the only place that work currently runs.
* **It hosts a stable, audited analysis contract that the product intentionally has not absorbed.** The MVP1 relational pipeline (strict 250-issue severity/clustering/fishbone analysis, reviewed nine-sheet Excel workbook) is deliberately kept as a separate, explicitly versioned sub-contract (`services/api/mvp1_contract.py`) rather than folded into the product API, specifically so its transport shape and its analysis logic can each evolve without the other silently drifting. See [MVP1 pipeline](/research/mvp1-pipeline).

## The hard boundary

The dependency direction is one-way and enforced structurally, not by convention alone:

* The repository's `Dockerfile` starts `uvicorn services.api.product_app:app`. Nothing in the deployed container ever imports `services.api.main`.
* `product_app.py`'s own module docstring states the reasoning directly: importing `main`'s app "creates stores and exposes operations the customer/staff product does not need."
* `product_app.py` imports exactly seven product routers — `auth_api`, `onboarding_api`, `insights_api`, `members_api`, `metrics_api`, `remediation_api`, `sources_api` — and never `services.api.main`, `services.api.clients_api`, `services.api.mvp1_contract`, or `causegraph.storage.memory.CauseLoopState`.
* `apps/web` is a separate Vite project from the Next.js tenant console in `frontend/`. It talks to `services/api/main.py` on its own local port (`5173` by default) and is never built into a deployed artifact.

<Note>
  If you are working on a product feature and find yourself reaching for something that only exists in `main.py` or `clients_api.py`, treat that as a signal, not a shortcut: either a product-side equivalent already exists (check `onboarding_api.py`, `remediation_api.py`, `sources_api.py` first) or it genuinely needs to be built on the product side — never imported from the research app. See [Research API](/api-reference/research-api#product-code-must-not-depend-on-this-surface) for the full statement of this rule.
</Note>

## Where to go next

* To run the workbench locally and choose the right mode for the work at hand, see [Research modes](/onboarding/research-modes).
* For a domain-by-domain tour of the \~279-path surface with representative endpoints, see [API surface](/research/api-surface).
* For the strict, OpenAI-governed 250-issue pipeline and its Postgres-backed reuse contract, see [MVP1 pipeline](/research/mvp1-pipeline).
* For what data is actually loaded versus what must be downloaded or generated, see [Datasets](/research/datasets).

## Related

* [API surface](/research/api-surface)
* [MVP1 pipeline](/research/mvp1-pipeline)
* [Datasets](/research/datasets)
* [Research API](/api-reference/research-api)