“Data policy” in Causeloop covers two genuinely different things that are easy to conflate because the staff portal presents them on the same onboarding step: a small, tenant-editable preference record (residency, retention, PII redaction), and a platform-wide object-storage lifecycle setting that is the same for every tenant and isn’t configured through this endpoint at all. This page keeps them separate.
The per-tenant data-policy record
Calls: GET / PUT /admin/tenants/{tenant_id}/data-policy, onboarding_admin for the PUT; any authenticated employee for the GET.
DataPolicyRequest (services/api/onboarding_api.py) validates both string fields against fixed tuples (_VALID_RESIDENCIES, _VALID_RETENTIONS) in model_post_init, raising a validation error for anything else. Storage mirrors training configuration’s pattern exactly: the whole object merges into the tenant’s own settings.key_values["data_policy"] JSONB column (PUT does a {**existing, "data_policy": policy} merge, so this write can never clobber training_config or any other key living in the same row) rather than a dedicated table — the same reasoning the code gives for training config applies here: it’s a small, tenant-scoped preference blob, not something that needs its own schema. GET reads the same row back, defaulting every field if the tenant has never set one.
A residency or retention value outside its fixed tuple raises a Pydantic validation error in DataPolicyRequest.model_post_init — surfaced as a 422 — before the route ever opens a tenant transaction.
What this record does not include
share_onboarding_outputs — whether onboarding-era insight collections are visible to the client before go-live — is not part of this JSONB blob. It’s its own column on control.tenants, set by POST .../activate, not by this endpoint. Both concepts show up together in the onboarding wizard’s flow (data policy in step 3, share-outputs in the final activation step), but they’re stored in different schemas by different routes with different authorization checks — don’t assume setting one affects the other.
Where each field is actually enforced
Of the three fields in the data-policy record, only one has a clearly wired enforcement path in the current codebase. Read this section carefully before assuming the portal’s residency/retention selectors have an operational effect.
residency — stored and returned; no code path in services/ reads this value to route storage, restrict a region, or gate any query. It is a recorded preference, not an enforced constraint, as far as this repository’s code shows.retention — same: stored and returned, and structurally distinct from the platform’s actual object-storage retention setting (below), which is not derived from this field.pii_redaction — the intent is that redacted text, not raw text, reaches any configured LLM. The actual field-level masking machinery exists — services/api/auth/pii_masking.py’s load_pii_policies()/mask_payload() reads pii_field_policies rows (dotted field path → redact/hash/partial) and redacts matching fields in a response payload for any caller lacking the pii.unmasked permission — but as of this writing, no router in services/api/ calls mask_payload() or load_pii_policies(). The masking utility is implemented and unit-testable; it is not currently invoked from any published request path. Treat the pii_redaction toggle as recording client-facing intent, not as an active enforcement switch, until a route calls it.
The three mask strategies mask_payload() understands (services/api/auth/pii_masking.py):
_PII_COLUMN_HINTS in onboarding_api.py maps detected column-name substrings to a suggested strategy — ssn/social_security/address/dob/date_of_birth/account_number/card_number suggest redact; email/phone/name suggest partial. A caller with the pii.unmasked permission (only org_admin’s seeded role profile grants it — see Client onboarding pipeline) bypasses masking entirely wherever mask_payload() is invoked.
The pii_field_policies rows this utility would consume are populated automatically — independently of the pii_redaction toggle — every time a dataset is committed: ingest/commit’s _detect_pii_columns() flags column names matching a PII heuristic (email, phone, ssn, name, address, dob, account_number, card_number, …) and inserts one pii_field_policies row per detected column with a suggested mask_strategy (ON CONFLICT DO NOTHING, so a manually adjusted policy from a prior commit is never overwritten). See Client onboarding pipeline for that detection step.
Separately from anything a tenant configures, every raw upload object in the shared object store expires on a fixed, platform-wide schedule controlled by CAUSELOOP_OBJECT_RETENTION_DAYS (default 365 days), applied identically across all deployment targets:
Locally this is a MinIO Information Lifecycle Management (ILM) rule on the single shared causeloop-uploads bucket — versioning is enabled on that bucket (mc version enable), and a separate rule expires noncurrent (superseded) object versions after a fixed 30 days regardless of the configured retention window. AWS and Azure apply the equivalent lifecycle policy at the storage-account/bucket level. Because every tenant’s raw uploads share one bucket (distinguished only by the tenants/<tenant_id>/uploads/... key prefix — see Client onboarding pipeline), this setting is a single platform-wide deployment parameter, not something any individual tenant’s retention selection changes. A tenant that picks "10 years" in its data-policy record does not get a longer-lived object-storage lifecycle than a tenant that picks "5 years" — both share the same bucket-level rule.
