> ## Documentation Index > Fetch the complete documentation index at: https://docs.causeloop.ai/llms.txt > Use this file to discover all available pages before exploring further. # MVP1 relational pipeline > The strict, governed 250-issue severity/clustering/root-cause pipeline: the explicit OpenAI transfer approval mechanism, content-addressed reuse in PostgreSQL, and the reviewed nine-sheet Excel workbook it produces. The MVP1 relational pipeline turns an immutable issue source into a governed, two-tier clustering workbook. It is the audited relational acceptance path for the 250-row banking and HomeCare acceptance datasets — the most heavily specified piece of the research surface, with its own migrations, its own content-addressed PostgreSQL runtime, and its own acceptance test suite. Source of truth: `docs/MVP1_RELATIONAL_PIPELINE.md` and `workers/run_banking_relational_e2e.py`. This is a worker-driven batch pipeline, not an HTTP endpoint. It is not exposed by any `/mvp1` route — the interactive `/mvp1/pipeline/run` and `/mvp1/issue-intelligence/latest` routes documented in [API surface](/research/api-surface#strict-mvp1-reports) use a separate, simpler interactive operational service and do not promise this pipeline's nine-sheet workbook or its issue-routed fishbone output. Treat the two as related but distinct: same underlying strict-analysis idea, different implementations, different output contracts. ## Pipeline stages Each issue moves through the following stages, from raw text to a validated, immutable report and workbook: ```mermaid theme={null} flowchart TD A["issue text
(id, unstructured_text, concern, cause, consequence)"] --> B["OpenAI eight-factor
severity matrix"] A --> C["OpenAI embedding
(text-embedding-3-small)"] B --> D["lexical + embedding + severity
similarities (equal weights)"] C --> D D --> E["full all-pairs
threshold graph"] E --> F["Tier 1 communities
(greedy modularity)"] F --> G["Tier 2 communities
inside each Tier 1 cluster
(greedy modularity)"] G --> H["OpenAI cluster names, summaries,
root-cause hypotheses, rationales"] H --> I["one canonical OpenAI fishbone route
per issue, using Tier 1 context"] I --> J["deterministic hard-hard, soft-hard,
and soft-soft allocations"] J --> K["OpenAI cause/rationale/remediation
synthesis per unique Tier-1/path set"] K --> L["immutable JSON report +
reviewed 9-sheet Excel workbook"] ``` The LLM generates all eight severity factors for each issue in one structured batch response. It does **not** generate issue type, product area, cost estimates, time estimates, or corrective actions for this strict profile — those enriched attributes exist elsewhere in the research surface but are never used for cluster membership here. ## Data and transfer boundary Only de-identified fields ever leave the process, and different stages see progressively less: | Stage | Fields sent externally | | ---------------------- | ----------------------------------------------------------------------------------------------------------------------------- | | Issue severity | Canonical issue ID, unstructured text, concern, cause, consequence | | Cluster interpretation | Derived titles and severity context for cluster members | | Issue routing | De-identified cause text and Tier 1 interpretation only — Tier 2 context and severity are excluded from the taxonomy decision | | Path synthesis | Deterministic path memberships, weights, compact cause signals, and Tier 1 interpretation only | Owner, customer/account identifiers, status, product area, and raw tags are excluded from every external call. The `OPENAI_API_KEY` used for these calls is loaded from the process environment (or an ignored local env file); it is never written to reports, workbooks, logs, or PostgreSQL payloads. ## The explicit transfer approval mechanism A live run requires two things together — a boolean flag is not sufficient on its own: 1. `--approve-openai-llm` on the command line. 2. `--openai-authorization ""` — a literal, dataset-specific string the runner compares byte-for-byte against a constant baked into the script. For the banking dataset, that constant is defined directly in `workers/run_banking_relational_e2e.py`: ```python theme={null} BANKING_OPENAI_AUTHORIZATION = ( "I explicitly authorize sending these 250 de-identified banking issue descriptions and derived " "severity context to the OpenAI API." ) ``` `_async_main` in that same script asserts both conditions before doing anything else — `args.approve_openai_llm` must be truthy, and `args.openai_authorization` must equal `BANKING_OPENAI_AUTHORIZATION` exactly. Passing the flag without the exact string (or a paraphrase of it) fails fast before any network call, migration, or database write. The HomeCare entrypoint (`workers/run_homecare_relational_e2e.py`) enforces the same two-part mechanism with its own dataset-specific authorization sentence. This is one of the explicit human-approval boundaries in `AGENTS.md`. A code agent must never construct or pass this authorization string on a user's behalf — the approval has to come from the user's own explicit, in-the-moment decision to incur OpenAI cost and transfer data, not be inferred from a task description. The runner also asserts its own client configuration before running, so a silent drift in batch size or model can't slip through unnoticed: the embedding provider must resolve to `text-embedding-3-small`; the severity client must use `batch_size=8` and `max_concurrency=8`; the cluster-label and fishbone clients must both use `max_concurrency=8`. ## Content-addressed reuse in PostgreSQL Alembic migrations `20260711_0002` through `20260714_0007` (`migrations/versions/`) create a normalized relational runtime purpose-built for this pipeline: * Pipeline runs, steps, and append-only events * Leased LLM batches, attempts, and batch items * One severity result plus exactly eight factor rows per issue * pgvector metadata and vectors, keyed by exact input hash * Two-tier clusters, with one membership per issue per tier * Content-addressed OpenAI cluster-label results and their per-run assignments * Content-addressed issue-route results, canonical leaf allocations, and per-run assignments * Content-addressed path-synthesis results, cause groups, exact evidence partitions, and per-run/variant assignments * Immutable report and artifact publications, with hashes and sizes Cache identity includes dataset, canonical issue or cluster identity, exact input hash, provider, model, prompt/schema contract, and required provenance. Each validated API response is committed immediately in its own small transaction, so a retry resumes only the work that's actually missing — an interrupted run never has to redo completed OpenAI calls. This is why a rerun on unchanged input is cheap and deterministic: an unchanged rerun must report ```text theme={null} severity_new_calls = 0 embedding_new_calls = 0 cluster_label_new_calls = 0 fishbone_route_new_calls = 0 fishbone_path_new_calls = 0 ``` The immutable report bytes are reproduced exactly, and a previously validated workbook is reused whenever the report hash and workbook hash are both unchanged — avoiding exporter ZIP-metadata churn and unnecessary artifact conflicts on a rerun that changed nothing. `packages/causegraph/src/causegraph/storage/mvp1_postgres.py` (`Mvp1PostgresRepository`) is the transactional repository behind all of this; `packages/causegraph/src/causegraph/providers/mvp1_llm.py`, `mvp1_cluster_labels.py`, and `mvp1_fishbone.py` are the batched OpenAI clients for severity, cluster labeling, and issue routing/path synthesis respectively. ## Reviewed Excel workbook contract The pipeline's output artifact is a reviewed nine-sheet workbook, built by `workers/artifacts/build_mvp1_clustering_workbook.mjs` (a Node script that imports the bundled `@oai/artifact-tool` package — see [Research modes](/onboarding/research-modes#mode-3-relational-db-to-excel) for the local Node-path setup this requires): 1. `Overview` 2. `Tier 1 Clusters` 3. `Tier 2 Clusters` 4. `Issue Assignments` 5. `Fishbone Allocations` 6. `Fishbone Paths` 7. `Cause & Remediation` 8. `Severity Matrix` 9. `LLM Audit` The builder validates the strict analysis contract before it will export: exact issue/factor coverage, one membership per issue per tier, one canonical route per issue, all three allocation variants (hard-hard, soft-hard, soft-soft), 100% per-issue allocation weights, weighted-issue reconciliation, exact cause-evidence coverage, provider provenance, formulas, sheet ranges, and post-export importability. It also renders every sheet for visual QA and includes a formula-backed stacked bar chart of weighted issue equivalents by variant and top-level bone. For a 250-row dataset, a complete acceptance run reconciles at least: 250 exact-input severity issues, 2,000 severity factor rows (8 per issue), 250 valid OpenAI embedding vectors, 250 Tier 1 memberships and 250 Tier 2 memberships, one label assignment and one unique response ID per cluster, 250 canonical issue routes and exactly 250 hard-hard allocations, 100% allocation weight for every issue under every variant, one path analysis per populated variant/Tier-1/exact-leaf path, exact cause-group evidence coverage for every path membership, and one published report plus the workbook and all nine sheet previews. ## Dataset entrypoints Both entrypoints share the same underlying runner logic (`workers/run_homecare_relational.py`'s `RelationalHomeCareRunner`, despite the module's name — it's the dataset-neutral relational execution engine both scripts drive) and apply Alembic before executing. The banking source defaults to the tracked synthetic snapshot at `docs/issue-set-2-records.json` — 250 records, no external file required: ```bash theme={null} uv run python workers/run_banking_relational_e2e.py \ --approve-openai-llm \ --openai-authorization \ "I explicitly authorize sending these 250 de-identified banking issue descriptions and derived severity context to the OpenAI API." ``` The same loader also accepts the original workbook via `--source`, but a fresh clone does not need it. HomeCare bootstraps from a workbook that is **not** tracked in the repository — `$HOME/Downloads/HomeCare Issue Set.xlsx` in the documented example is a machine-specific path, not something a fresh clone will have: ```bash theme={null} .venv/bin/python workers/run_homecare_relational_e2e.py \ --source "$HOME/Downloads/HomeCare Issue Set.xlsx" \ --approve-openai-llm \ --openai-authorization \ "I explicitly authorize sending these 250 de-identified HomeCare issue descriptions and derived severity context to the OpenAI API." ``` It bootstraps a checksum-ledgered historical checkpoint before requesting only missing work, so a source file that hasn't changed doesn't trigger a full recompute either. The 250-record file's own internal `dataset` field is `cause_loop_issue_set_2` — the same identifier the interactive `/mvp1` product routes use as their `dataset_id` default (see `docs/MVP1_API_CONTRACT.md`). The relational banking runner, however, uses its own Postgres content-addressing identifier, `DATASET_ID = "banking_issue_set_250"`, and its own tenant scope (`TENANT_ID = "demo"`). These are two different identifiers for two different consumers of the same 250 rows — don't assume one implies the other when cross-referencing the interactive service and this relational pipeline. ## Output layout Output defaults to the repository-relative `outputs/mvp1/` directory. It is untracked by convention — nothing in the codebase creates or commits files there — but it is **not** covered by `.gitignore`, so nothing structurally prevents a run's output from being accidentally `git add`ed: ```text theme={null} outputs/mvp1/ ├── banking_openai_relational_report.json # immutable analysis report (--report-output) ├── Banking_Tier1_Tier2_Clustering_OpenAI_Strict.xlsx # reviewed workbook (--workbook-output) └── previews-banking-strict/ # per-sheet visual QA renders (--preview-dir) ``` Every path — report, workbook, preview directory, workbook builder script, and Node executable — is overridable with the corresponding CLI flag, so a custom local layout never requires editing the script itself. ## Acceptance checks Focused regression tests cover the runner, the acceptance contract, checkpoint migration, cluster labeling, fishbone routing, LLM batching, Postgres persistence, and the strict pipeline end to end: ```bash theme={null} .venv/bin/pytest -q \ tests/test_homecare_relational_runner.py \ tests/test_homecare_relational_e2e.py \ tests/test_banking_relational_runner.py \ tests/test_mvp1_acceptance_contract.py \ tests/test_mvp1_checkpoint_migration.py \ tests/test_mvp1_cluster_labels.py \ tests/test_mvp1_fishbone.py \ tests/test_mvp1_enrichment_parallelism.py \ tests/test_mvp1_llm.py \ tests/test_mvp1_postgres.py \ tests/test_mvp1_relational_severity_report.py \ tests/test_mvp1_strict_pipeline.py \ tests/test_mvp1_workflow.py \ tests/test_mvp1_issue_intelligence.py ``` ## Key implementation files | File | Role | | --------------------------------------------------------------------- | ------------------------------------------------------------------------------------ | | `workers/run_homecare_relational.py` | Dataset-neutral relational runtime (`RelationalHomeCareRunner`, `OutputPaths`) | | `workers/run_homecare_relational_e2e.py` | HomeCare bootstrap orchestration | | `workers/run_banking_relational_e2e.py` | Banking source adapter and orchestration | | `workers/artifacts/build_mvp1_clustering_workbook.mjs` | Reviewed workbook builder (Node, `@oai/artifact-tool`) | | `workers/migrate_homecare_checkpoint.py` | One-time, zero-recompute checkpoint migration | | `packages/causegraph/src/causegraph/storage/mvp1_postgres.py` | Transactional repository (`Mvp1PostgresRepository`) | | `packages/causegraph/src/causegraph/providers/mvp1_llm.py` | Batched severity client | | `packages/causegraph/src/causegraph/providers/mvp1_cluster_labels.py` | Cluster label client | | `packages/causegraph/src/causegraph/providers/mvp1_fishbone.py` | Issue routing, deterministic allocation variants, path synthesis, v5 report contract | ## Related * [Research modes](/onboarding/research-modes) * [API surface](/research/api-surface) * [Datasets](/research/datasets) * [Research workbench overview](/research/overview)