> ## Documentation Index
> Fetch the complete documentation index at: https://docs.causeloop.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Data policy

> The per-tenant data-policy blob (residency, retention, PII redaction), its relationship to share_onboarding_outputs, the platform's object-storage retention setting, and where each is actually enforced.

"Data policy" in Causeloop covers two genuinely different things that are easy to conflate because the staff portal presents them on the same onboarding step: a small, tenant-editable **preference record** (residency, retention, PII redaction), and a **platform-wide object-storage lifecycle setting** that is the same for every tenant and isn't configured through this endpoint at all. This page keeps them separate.

## The per-tenant data-policy record

**Calls:** `GET` / `PUT /admin/tenants/{tenant_id}/data-policy`, `onboarding_admin` for the `PUT`; any authenticated employee for the `GET`.

```json theme={null}
{
  "residency": "US",
  "retention": "7 years",
  "pii_redaction": true
}
```

| Field           | Valid values                     | Default   |
| --------------- | -------------------------------- | --------- |
| `residency`     | `US`, `EU`, `APAC`               | `US`      |
| `retention`     | `5 years`, `7 years`, `10 years` | `7 years` |
| `pii_redaction` | boolean                          | `true`    |

`DataPolicyRequest` (`services/api/onboarding_api.py`) validates both string fields against fixed tuples (`_VALID_RESIDENCIES`, `_VALID_RETENTIONS`) in `model_post_init`, raising a validation error for anything else. Storage mirrors [training configuration](/features/training-and-checkpoints)'s pattern exactly: the whole object merges into the tenant's own `settings.key_values["data_policy"]` JSONB column (`PUT` does a `{**existing, "data_policy": policy}` merge, so this write can never clobber `training_config` or any other key living in the same row) rather than a dedicated table — the same reasoning the code gives for training config applies here: it's a small, tenant-scoped preference blob, not something that needs its own schema. `GET` reads the same row back, defaulting every field if the tenant has never set one.

```bash theme={null}
curl -sX PUT http://localhost:18000/admin/tenants/$TENANT_ID/data-policy \
  -H 'content-type: application/json' -H "x-csrf-token: $CSRF" \
  --cookie "cl_admin_session=$SESSION; cl_csrf=$CSRF" \
  -d '{"residency": "EU", "retention": "10 years", "pii_redaction": true}'
# -> {"status": "ok", "residency": "EU", "retention": "10 years", "pii_redaction": true}

curl -s http://localhost:18000/admin/tenants/$TENANT_ID/data-policy \
  --cookie "cl_admin_session=$SESSION"
# -> {"residency": "EU", "retention": "10 years", "pii_redaction": true}
```

A `residency` or `retention` value outside its fixed tuple raises a Pydantic validation error in `DataPolicyRequest.model_post_init` — surfaced as a `422` — before the route ever opens a tenant transaction.

### What this record does *not* include

`share_onboarding_outputs` — whether onboarding-era insight collections are visible to the client before go-live — is **not** part of this JSONB blob. It's its own column on `control.tenants`, set by [`POST .../activate`](/features/training-and-checkpoints#activation-semantics), not by this endpoint. Both concepts show up together in the onboarding wizard's flow (data policy in step 3, share-outputs in the final activation step), but they're stored in different schemas by different routes with different authorization checks — don't assume setting one affects the other.

## Where each field is actually enforced

<Warning>
  Of the three fields in the data-policy record, only one has a clearly wired enforcement path in the current codebase. Read this section carefully before assuming the portal's residency/retention selectors have an operational effect.
</Warning>

* **`residency`** — stored and returned; no code path in `services/` reads this value to route storage, restrict a region, or gate any query. It is a recorded preference, not an enforced constraint, as far as this repository's code shows.
* **`retention`** — same: stored and returned, and structurally distinct from the platform's actual object-storage retention setting (below), which is not derived from this field.
* **`pii_redaction`** — the intent is that redacted text, not raw text, reaches any configured LLM. The actual field-level masking machinery exists — `services/api/auth/pii_masking.py`'s `load_pii_policies()`/`mask_payload()` reads `pii_field_policies` rows (dotted field path → `redact`/`hash`/`partial`) and redacts matching fields in a response payload for any caller lacking the `pii.unmasked` permission — but as of this writing, no router in `services/api/` calls `mask_payload()` or `load_pii_policies()`. The masking utility is implemented and unit-testable; it is not currently invoked from any published request path. Treat the `pii_redaction` toggle as recording client-facing intent, not as an active enforcement switch, until a route calls it.

The three mask strategies `mask_payload()` understands (`services/api/auth/pii_masking.py`):

| Strategy  | Transform                                                                                      |
| --------- | ---------------------------------------------------------------------------------------------- |
| `redact`  | Replaces the value with the literal string `***`                                               |
| `hash`    | SHA-256 of the value, truncated to 16 hex characters                                           |
| `partial` | Keeps the first and last 2 characters, masks the middle (`"*" * len` if 4 characters or fewer) |

`_PII_COLUMN_HINTS` in `onboarding_api.py` maps detected column-name substrings to a suggested strategy — `ssn`/`social_security`/`address`/`dob`/`date_of_birth`/`account_number`/`card_number` suggest `redact`; `email`/`phone`/`name` suggest `partial`. A caller with the `pii.unmasked` permission (only `org_admin`'s seeded role profile grants it — see [Client onboarding pipeline](/features/client-onboarding-pipeline#2-provision--draftonboarding--onboarding)) bypasses masking entirely wherever `mask_payload()` is invoked.

The `pii_field_policies` rows this utility would consume *are* populated automatically — independently of the `pii_redaction` toggle — every time a dataset is committed: `ingest/commit`'s `_detect_pii_columns()` flags column names matching a PII heuristic (`email`, `phone`, `ssn`, `name`, `address`, `dob`, `account_number`, `card_number`, …) and inserts one `pii_field_policies` row per detected column with a suggested `mask_strategy` (`ON CONFLICT DO NOTHING`, so a manually adjusted policy from a prior commit is never overwritten). See [Client onboarding pipeline](/features/client-onboarding-pipeline#3-historical-data-ingest--upload-then-commit) for that detection step.

## Platform-wide object-storage retention

Separately from anything a tenant configures, every raw upload object in the shared object store expires on a fixed, platform-wide schedule controlled by `CAUSELOOP_OBJECT_RETENTION_DAYS` (default **365 days**), applied identically across all deployment targets:

<CodeGroup>
  ```yaml Local (infra/docker-compose.product.yml) theme={null}
  minio-init:
    environment:
      CAUSELOOP_OBJECT_RETENTION_DAYS: ${CAUSELOOP_OBJECT_RETENTION_DAYS:-365}
    entrypoint:
      - /bin/sh
      - -ec
      - |
        mc alias set local http://minio:9000 causeloop causeloop-dev
        mc mb --ignore-existing local/causeloop-uploads
        mc anonymous set none local/causeloop-uploads
        mc version enable local/causeloop-uploads
        mc ilm rule remove --all --force local/causeloop-uploads || true
        mc ilm rule add \
          --expire-days "$${CAUSELOOP_OBJECT_RETENTION_DAYS}" \
          --noncurrent-expire-days 30 \
          local/causeloop-uploads
  ```

  ```hcl AWS (deploy/aws/variables.tf) theme={null}
  variable "object_retention_days" {
    type    = number
    default = 365
  }
  ```

  ```bicep Azure (deploy/azure/main.bicep) theme={null}
  param objectRetentionDays int = 365
  // applied via a 'tenant-object-retention' lifecycle management rule:
  // daysAfterModificationGreaterThan: objectRetentionDays
  ```
</CodeGroup>

Locally this is a MinIO Information Lifecycle Management (ILM) rule on the single shared `causeloop-uploads` bucket — versioning is enabled on that bucket (`mc version enable`), and a **separate** rule expires noncurrent (superseded) object versions after a fixed 30 days regardless of the configured retention window. AWS and Azure apply the equivalent lifecycle policy at the storage-account/bucket level. Because every tenant's raw uploads share one bucket (distinguished only by the `tenants/<tenant_id>/uploads/...` key prefix — see [Client onboarding pipeline](/features/client-onboarding-pipeline#3-historical-data-ingest--upload-then-commit)), this setting is a single platform-wide deployment parameter, not something any individual tenant's `retention` selection changes. A tenant that picks `"10 years"` in its data-policy record does not get a longer-lived object-storage lifecycle than a tenant that picks `"5 years"` — both share the same bucket-level rule.

## Related

* [Client onboarding pipeline](/features/client-onboarding-pipeline)
* [Training and checkpoints](/features/training-and-checkpoints)
* [Security model](/architecture/security)
* [Tenancy and data model](/architecture/tenancy-and-data-model)