> ## Documentation Index
> Fetch the complete documentation index at: https://docs.causeloop.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Local onboarding walkthrough

> An end-to-end dev test of the client onboarding pipeline against the local product stack, with the API call, worker job, and psql check behind every step.

This walks the full onboarding pipeline — tenant creation through a live client sign-in — against [the local product stack](/onboarding/local-product-stack). Every step below names the exact UI action, the backend route it calls (`services/api/onboarding_api.py` unless noted), which durable worker consumes the resulting job, and a `psql` query to confirm it landed. Bring the stack up first:

```bash theme={null}
docker compose -f infra/docker-compose.product.yml up -d --build
python3 scripts/smoke_product.py
```

<Note>
  Tenant creation only happens from the **New client** wizard (`/admin/tenants/new`). Once the tenant exists it appears in the **Clients** directory in any lifecycle state, and clicking into it opens its own detail page (`/admin/tenants/{tenantId}`) with the remaining steps as independent actions — that per-step page, not the wizard, is what this walkthrough narrates from Provision onward, because each action there maps 1:1 to one backend call you can verify in isolation. The wizard offers a faster, linear path through most of the same calls — plus two the detail-page path never touches, `PUT .../data-policy` and `POST .../sources/seed` — see the callout at the end of Activate for exactly how it differs.
</Note>

All `psql` checks below run through the Compose Postgres container:

```bash theme={null}
docker compose -f infra/docker-compose.product.yml exec postgres \
  psql -U causeloop -d causeloop -c "<query>"
```

## Tenant lifecycle

`control.tenants.lifecycle_state` is the state this whole walkthrough drives forward. The transitions below are reachable through this pipeline (`services/api/tenancy/provisioner.py`, `services/api/onboarding_api.py`):

```mermaid theme={null}
stateDiagram-v2
    [*] --> draft: create_tenant (POST /admin/tenants)
    draft --> onboarding: provision_tenant job's activate_onboarding step
    onboarding --> review: activate_checkpoint (POST .../activate)
    review --> live: go_live (POST .../go-live)
    live --> review: activate_checkpoint (POST .../activate)
```

<Info>
  `provision` (`POST .../provision`) also accepts a tenant already in a `provisioning` state as a no-op guard, but no code path in this repository ever writes that value — every tenant observed here goes `draft` straight to `onboarding` once its provisioning job's last step (`activate_onboarding`) completes.
</Info>

<Warning>
  `activate_checkpoint` has no lifecycle guard: it unconditionally sets `lifecycle_state = 'review'`, even when the tenant is already `live`. Re-activating a checkpoint on a live tenant (for example, to ship a retrained model) silently knocks it back to `review` — and `login_tenant_user` requires `lifecycle_state == 'live'`, so client sign-in breaks until someone clicks **Go live** again. There is no confirmation prompt for this.
</Warning>

<Steps>
  <Step title="Staff sign-in">
    Open `http://127.0.0.1:13000/login`, select the **Causeloop Employee** tab, and sign in with the local-only credential `admin@causeloop.local` / `Local-Admin-Pass1!` (seeded by the `local-admin` Compose service — see [Local product stack](/onboarding/local-product-stack)). You land on `/admin`, the Clients directory.

    * **Endpoint:** `POST /admin/login` (`services/api/auth_api.py`), backed by `login_employee` in `services/api/auth/employee_auth.py`.
    * **Verify:**

    ```bash theme={null}
    docker compose -f infra/docker-compose.product.yml exec postgres \
      psql -U causeloop -d causeloop -c \
      "SELECT email, control_role, is_active FROM control.employees;"
    ```
  </Step>

  <Step title="Create the tenant">
    In the rail, click **New client**, fill in **Organization name** (the slug auto-derives via `slugify()` unless you edit it directly), industry, region, and a primary admin email, then **Continue →**. This is the only step that runs inside the wizard — it creates the draft control-plane row and nothing else; no tenant schema exists yet.

    * **Endpoint:** `POST /admin/tenants` → `create_tenant`, which validates the slug (`validate_tenant_slug`, lowercase alphanumeric + hyphens, 3–63 chars) and calls `create_draft_tenant` (`services/api/tenancy/provisioner.py`).
    * **Verify:**

    ```sql theme={null}
    SELECT id, slug, name, lifecycle_state, schema_name, storage_prefix
    FROM control.tenants WHERE slug = 'acme';
    -- lifecycle_state = 'draft', schema_name = 'tenant_acme'
    ```

    Now leave the wizard — click **Clients** in the rail, then click the new tenant's row to open its detail page. The rest of this walkthrough happens there.

    <Note>
      This walkthrough uses `acme` as the example slug throughout. `tenant_schema_name()` (`services/api/tenancy/db.py`) builds the schema name as `f"tenant_{slug}"`, preserving any hyphens in the slug verbatim — so a slug like `your-slug` gets the schema `tenant_your-slug`, which requires double-quoting (`SET search_path TO "tenant_your-slug"`) because Postgres identifiers can't contain a bare hyphen otherwise. A hyphen-free slug avoids that quoting entirely.
    </Note>
  </Step>

  <Step title="Provision">
    On the tenant detail page, **Step 1. Provision** shows a **Provision tenant** button. Click it and watch `provisioner-worker`'s log:

    ```bash theme={null}
    docker compose -f infra/docker-compose.product.yml logs -f provisioner-worker
    ```

    * **Endpoint:** `POST /admin/tenants/{tenant_id}/provision` → `ensure_provision_job`, which synchronously inserts a `control.platform_jobs` row (`job_type='provision_tenant'`, `status='queued'`) before any background work starts — `provisioner-worker` flips it to `running` when it claims the job — so the immediate next poll never reads a stale `not_started`.
    * **Worker job:** `provision_tenant`, consumed only by `provisioner-worker` (the only worker holding owner-level `MIGRATIONS_DATABASE_URL`). `provision_tenant()` runs six idempotent, individually-tracked steps: `create_schema`, `run_tenant_migrations`, `grant_app_rw`, `seed_role_profiles`, `verify_object_storage`, `activate_onboarding`. The last step flips `control.tenants.lifecycle_state` from `draft`/`provisioning` to `onboarding`.
    * **Verify:**

    ```sql theme={null}
    SELECT job_type, status, progress, steps_completed, error_detail
    FROM control.platform_jobs
    WHERE tenant_id = (SELECT id FROM control.tenants WHERE slug = 'acme')
      AND job_type = 'provision_tenant'
    ORDER BY created_at DESC LIMIT 1;
    -- status = 'succeeded', steps_completed has all 6 entries
    SELECT lifecycle_state, schema_name FROM control.tenants WHERE slug = 'acme';
    -- lifecycle_state = 'onboarding'
    ```

    The detail page polls `GET /admin/tenants/{tenant_id}/provision/status` every 2s while `running` and shows the tenant's Postgres schema name once done.

    <Warning>
      The **New client** wizard's own provisioning checklist (step "Environment") has a cosmetic bug worth knowing about if you use that path instead: its `PROVISION_STEPS` array (`frontend/src/app/admin/(portal)/tenants/new/page.tsx`) labels the fifth row `create_storage_prefix`, but the backend never writes that key to `steps_completed` — it writes `verify_object_storage`. That one checklist row (and the "running" pulse on the row after it) never lights up even after provisioning fully succeeds. Trust the **Provisioned ✓** button state and the **Continue →** button being enabled, not every individual checklist row.
    </Warning>
  </Step>

  <Step title="Upload historical data">
    **Step 2. Historical data ingest** now shows a file picker. Choose a CSV, XLSX, JSON, or JSONL file (`packages/causegraph/src/causegraph/onboarding/ingestion.py` dispatches on the extension) — a representative issue/finding register works well. The API responds with a row count, a suggested column mapping (`issue_id`, `text`, `created_at`), and any columns it heuristically flags as PII (matched against hints like `email`, `ssn`, `account_number` in `onboarding_api.py`'s `_PII_COLUMN_HINTS`). Review the mapping, then click **Commit dataset v1**.

    * **Endpoints:** `POST .../ingest/upload` (stores the raw file in MinIO under the tenant's storage prefix, versioned, and parses/previews it — nothing is committed to Postgres yet) then `POST .../ingest/commit` (normalizes rows per the mapping and writes `datasets` + `records` rows in the tenant schema).
    * **Worker job:** committing enqueues `materialize_insights` (`enqueue_platform_job`, idempotency key `materialize:{tenant_id}:{dataset_id}:{version}`), consumed by `product-worker`. This is what turns raw records into the insight collections the client console reads later.
    * **Verify** (tenant-schema tables are unqualified, so set the search path first):

    ```sql theme={null}
    SET search_path TO tenant_acme;
    SELECT version, row_count FROM datasets ORDER BY version DESC LIMIT 1;
    SELECT count(*) FROM records;
    ```

    ```sql theme={null}
    SELECT job_type, status FROM control.platform_jobs
    WHERE job_type = 'materialize_insights' ORDER BY created_at DESC LIMIT 1;
    ```
  </Step>

  <Step title="Training configuration">
    **Step 3. Training configuration** is fixed to the deterministic, offline embedding provider (`hash`, dimension 64) and lets you pick the clustering algorithm (`hdbscan`, `kmeans`, or `mixed_graph_greedy_modularity`). LLM narrative enrichment is disabled in this runtime. Click **Save configuration**.

    * **Endpoint:** `PUT .../training-config` — upserts `tenant_schema.settings.key_values->'training_config'` and a matching `llm_settings` row (any API key you set here is Fernet-encrypted via `encrypt_secret`/`SOURCE_CREDS_KEY`; only its hint is ever returned).
    * **Verify:**

    ```sql theme={null}
    SET search_path TO tenant_acme;
    SELECT key_values->'training_config' FROM settings;
    ```
  </Step>

  <Step title="Train">
    **Step 4. Train** shows a **Train model** button once a dataset exists.

    * **Endpoint:** `POST .../train` → looks up the tenant's latest dataset and saved training config, hashes the config, and enqueues `train_tenant_model` with idempotency key `train:{tenant_id}:{dataset_id}:{config_hash}` (`retry_terminal=True`, so a fixed and rerun job is allowed to run again even from a terminal `failed` state).
    * **Worker job:** `train_tenant_model`, consumed by `product-worker`.
    * **Verify:**

    ```sql theme={null}
    SELECT status, progress, error_detail FROM control.platform_jobs
    WHERE tenant_id = (SELECT id FROM control.tenants WHERE slug = 'acme')
      AND job_type = 'train_tenant_model' ORDER BY created_at DESC LIMIT 1;
    ```

    The detail page polls `GET .../train/status` every 2s while `running` and invalidates the checkpoints list the moment it reports `succeeded`.
  </Step>

  <Step title="Checkpoint & activate">
    Once training succeeds, **Step 5. Activation & output sharing** lists every `model_checkpoints` row (version, algorithm, cluster count). Click **Activate** on the version you want live, and set **Share onboarding outputs with the client** — this decides whether the client sees this training run's results on first login or starts from an empty dashboard.

    * **Endpoint:** `POST .../activate` (`ActivateCheckpointRequest{checkpoint_version, share_onboarding_outputs}`) — deactivates every other checkpoint, activates the chosen one, re-enqueues `materialize_insights` against the current dataset with `client_visible` set from your choice, and sets `control.tenants.lifecycle_state = 'review'`.
    * **Verify:**

    ```sql theme={null}
    SET search_path TO tenant_acme;
    SELECT version, algorithm, is_active FROM model_checkpoints ORDER BY version DESC;
    ```

    ```sql theme={null}
    SELECT lifecycle_state, share_onboarding_outputs FROM control.tenants WHERE slug = 'acme';
    -- lifecycle_state = 'review'
    ```

    <Note>
      The **New client** wizard's final step bundles this Activate call together with Invite and Go live into one **Go live →** click, using the admin email you typed on the very first wizard screen and a hardcoded `org_admin` role. It does **not** wait for that invite to be accepted before going live. Everything downstream in this walkthrough still applies either way — only the order in which you personally click things changes.
    </Note>
  </Step>

  <Step title="Invite a user">
    **Step 6. Invite users** takes an email and a role profile (`org_admin`, `risk_admin`, `analyst`, or `viewer` — the default role profiles `provision_tenant` seeded per tenant). Submit **Invite**.

    * **Endpoint:** `POST .../invite` → `create_invitation` (`services/api/auth/tenant_auth.py`) writes a `control.tenant_invitations` row (token hashed, 7-day expiry) and enqueues a `send_email` job with the accept-invite link.
    * **Worker job:** `send_email`, consumed by `product-worker`, delivered over SMTP to Mailpit.
    * **Verify:**

    ```sql theme={null}
    SELECT email, role_profile_key, status, expires_at
    FROM control.tenant_invitations
    WHERE tenant_id = (SELECT id FROM control.tenants WHERE slug = 'acme')
    ORDER BY created_at DESC LIMIT 1;
    -- status = 'pending'
    ```
  </Step>

  <Step title="Accept the invite in Mailpit">
    Open `http://127.0.0.1:18025`, find the message ("You're invited to `acme` on CauseLoop"), and open its accept-invite link (`/accept-invite?workspace=acme&token=...`). Set a full name (optional) and a password (8+ characters), then submit.

    * **Endpoint:** `POST /accept-invite` (`services/api/auth_api.py` → `accept_invitation`) — creates (or reactivates) the `TenantUser` row, assigns the invited role profile, marks the invitation `accepted`, then immediately attempts to log the new user in.
    * **Verify:**

    ```sql theme={null}
    SELECT status, accepted_at FROM control.tenant_invitations WHERE email = 'invited@client.com';
    ```

    ```sql theme={null}
    SET search_path TO tenant_acme;
    SELECT email, is_active FROM users WHERE email = 'invited@client.com';
    ```

    <Warning>
      **Expected stall — auto-login 401 before go-live.** `accept_invitation`'s final step calls the normal tenant login path, which rejects any tenant whose `lifecycle_state` isn't `live`. If you accept the invite before clicking Go live (as this walkthrough does), the account is created and the invitation **is** marked accepted — but the frontend shows *"Your account was created, but this workspace isn't live yet."* This is not a bug; it's `login_tenant_user`'s `lifecycle_state != "live"` guard. Continue to Go live, then have the client sign in normally at `/login`.
    </Warning>
  </Step>

  <Step title="Go live">
    Back on the tenant detail page, **Step 7. Go live**'s button is clickable as soon as the tenant is provisioned — the UI never gates it client-side. Its three real preconditions are enforced only server-side, checked live by `POST .../go-live`:

    * at least one dataset ingested,
    * an active model checkpoint,
    * at least one invitation in `pending` or `accepted` status (acceptance is **not** required — a pending invite is enough).

    Click **Go live**. Any missing precondition returns `409` naming exactly what's missing (`"Cannot go live yet: <missing items>."`).

    * **Endpoint:** `POST .../go-live` → sets `control.tenants.lifecycle_state = 'live'`.
    * **Verify:**

    ```sql theme={null}
    SELECT lifecycle_state FROM control.tenants WHERE slug = 'acme';
    -- lifecycle_state = 'live'
    ```
  </Step>

  <Step title="Open the portal, or sign in as the client">
    Back in the **Clients** directory, the tenant now shows a **Live** chip and an **Open portal →** button.

    * **Open portal (impersonate):** `POST .../impersonate` mints a real `cl_tenant_session` cookie for the tenant's **earliest-created active user** and redirects to `/` — this is a genuine access grant, not a read-only preview, so it is unconditionally audit-logged (`control.audit_log`, action `employee_impersonation`) regardless of whether anyone reviews that log.
    * **Or sign in directly:** open `http://127.0.0.1:13000/login`, keep the default **Client sign-in** tab, and enter the workspace (tenant slug) plus the email/password set in the Accept-invite step.
    * **Verify:**

    ```sql theme={null}
    SELECT action, resource_type, detail, created_at FROM control.audit_log
    WHERE tenant_id = (SELECT id FROM control.tenants WHERE slug = 'acme')
    ORDER BY created_at DESC LIMIT 5;
    ```

    <Warning>
      **"Tenant has no active users to impersonate."** — `impersonate_tenant_user` (`services/api/auth/tenant_auth.py`) returns `409` with exactly this message whenever a live tenant has zero `TenantUser` rows with `is_active = true`. Since Go live only requires an invitation to *exist* (pending is enough — see the previous step), it is entirely possible to go live before anyone has actually accepted an invite. If you hit this, either wait for the invited user to complete the Accept-invite step, or invite and accept a second user first.
    </Warning>
  </Step>
</Steps>

## Other common stalls

<AccordionGroup>
  <Accordion title="The console never leaves its loading/locked skeleton">
    Check `GET /health/ready` directly (`curl http://127.0.0.1:18000/health/ready` or through the proxy at `http://127.0.0.1:13000/api/backend/health/ready`). It names exactly which dependency — `database`, `worker`, `object_storage`, `credential_encryption`, `email_delivery`, or `redis` — is failing. `worker` most commonly means one of the two durable workers hasn't written a `control.runtime_heartbeats` row in the last 45 seconds; check `docker compose ... logs product-worker provisioner-worker` for a crash loop.
  </Accordion>

  <Accordion title="Provisioning appears stuck at 'not started'">
    A poll landing before the provisioning job's first database write reads `not_started`, not `running` — `ensure_provision_job` closes this race at the source by inserting the job row synchronously before returning `202`. The **New client** wizard (`tenants/new/page.tsx`) keeps polling on `not_started` as a defensive backstop, but the tenant detail page (`tenants/[tenantId]/page.tsx`) only refetches while status is `running` — a poll that lands on `not_started` there stops polling outright, so a manual refresh may be needed. If it's still `not_started` after several seconds, check `provisioner-worker`'s logs directly; it may not have started (see [Local product stack](/onboarding/local-product-stack)'s startup-order diagram — it depends on `migrate` and `minio-init` completing).
  </Accordion>

  <Accordion title="Upload returns 503">
    `ingest/upload` raises `503 Object storage is unavailable` (`ObjectStoreError`) if MinIO isn't reachable or `minio-init` hasn't finished creating/versioning the `causeloop-uploads` bucket yet. Confirm with `docker compose ... ps minio minio-init` — `minio-init` should show `Exit 0`, not still running or failing.
  </Accordion>

  <Accordion title="Ingest commit returns 422">
    `IngestError` (unsupported file extension, empty workbook/CSV, invalid JSON, or a mapping that points at a non-existent column) surfaces as `422` with the specific reason in `detail`. The failed upload is marked `status='failed'` (`mark_upload_failed`) rather than left ambiguous — re-upload a corrected file rather than retrying the same `upload_id`.
  </Accordion>

  <Accordion title="Invitation shows 'email missing' in the tenant detail page's list">
    `list_invitations` looks up each invitation's delivery job by idempotency key (`email:tenant-invitation:{invitation_id}`); `"missing"` means no matching `send_email` job row was ever found in `control.platform_jobs`. This can't be caused by a stopped `product-worker` or an unhealthy Mailpit — `create_invitation` enqueues the `send_email` job in the same transaction as the invitation row, so a healthy invite always has at least a `queued` or `failed` delivery job. `"missing"` only happens if the job row itself was removed after the fact (or the invitation predates the delivery-job wiring).
  </Accordion>
</AccordionGroup>

## Related

* [Local product stack](/onboarding/local-product-stack)
* [Client onboarding pipeline](/features/client-onboarding-pipeline)
* [Invitations and impersonation](/features/invitations-and-impersonation)
* [Workers and jobs](/architecture/workers-and-jobs)