Tenancy - Lifecycle and workflows¶
- Status: Draft
- Prefix:
TLF - Module:
libs/api/tenancy(@bizos/api-tenancy) - Feature:
lifecycle - Related: Tenancy core, Tenant management, Provisioning monitor, Backup, restore, and reset, Tenant sign-up, ADR-0012
Purpose¶
This specification defines the complete end-to-end lifecycle, finite state machine, and operational workflows of a tenant in BizOS: from customer sign-up, automated provisioning, regular day-to-day operations and maintenance, through suspension, scheduled deletion, grace period handling, and ultimate hard teardown and data retention.
Users and permissions¶
- Prospective Customer / Tenant Owner: Registers on the landing page, completes the sign-up wizard, operates within the active tenant workspace.
- Tenant Members & Managers: Access services only while the tenant is in
Activestatus. - Operator & Owner (Platform Ops): Monitor provisioning runs, trigger maintenance (backup, restore, reset), suspend and reactivate tenants.
- Platform Owner (Super Admin): Only role allowed to schedule deletion or cancel pending deletion of a tenant.
State machine diagram¶
The tenant lifecycle is governed by an explicit finite state machine with 7 primary states. Transitions preserve previous_state where necessary to support rollbacks (e.g. exiting maintenance or cancelling a scheduled deletion).
stateDiagram-v2
[*] --> Provisioning: Sign-up Wizard Completed
state Provisioning {
[*] --> RegisterTenant
RegisterTenant --> CreateDatabase
CreateDatabase --> ApplySchema
ApplySchema --> SeedConfig
SeedConfig --> CreateOwner
CreateOwner --> Activate
Activate --> SendWelcomeEmail
SendWelcomeEmail --> [*]
}
Provisioning --> Active: Run Succeeds / Activated
Provisioning --> ProvisioningFailed: Run Fails (after 3 retries)
ProvisioningFailed --> Provisioning: Operator / Automated Retry
state OperationalLifecycle {
Active --> Maintenance: Backup / Restore / Reset Triggered
Maintenance --> Active: Action Finished (Success/Fail) & Previous = Active
Active --> Suspended: Suspend (ToS Breach / Non-payment / Manual)
Suspended --> Active: Reactivate (Reason >= 10 chars)
Suspended --> Maintenance: Restore / Reset Triggered on Suspended Tenant
Maintenance --> Suspended: Action Finished & Previous = Suspended
}
Active --> PendingDeletion: Schedule Deletion (Owner only + Subdomain confirmation)
Suspended --> PendingDeletion: Schedule Deletion (Owner only + Subdomain confirmation)
ProvisioningFailed --> PendingDeletion: Schedule Deletion (Clean abandoned setup)
PendingDeletion --> Active: Cancel Deletion (if Previous = Active)
PendingDeletion --> Suspended: Cancel Deletion (if Previous = Suspended)
PendingDeletion --> Deleted: Grace Period Expired (30 days)
Deleted --> [*]: Subdomain Hold Expired (90 days) & Audit Record Preserved
Complete End-to-End Tenancy Lifecycle Flowchart¶
flowchart TD
%% -------------------------------------------------------------
%% PHASE 1: ONBOARDING & PROVISIONING
%% -------------------------------------------------------------
subgraph Phase1["Phase 1: Onboarding & Provisioning Pipeline"]
A1(["1. Customer Registers"]) --> A2["2. Select Plan<br/>Starter / Standard / Advanced"]
A2 --> A3["3. Enter Company Info & Subdomain<br/>Validate Reserved & Unique Subdomain"]
A3 --> A4["4. Checkout & Payment Confirmation"]
A4 --> A5["5. Insert Control-Plane Tenant Registry<br/>State: Provisioning"]
A5 --> P1{"7-Step Provisioning Pipeline"}
P1 -->|Step 1-2| P2["Register Tenant & Create DB"]
P2 -->|Step 3-4| P3["Apply Migrations & Seed Default Config"]
P3 -->|Step 5-7| P4["Create Owner Account, Activate & Send Welcome Email"]
P1 -.->|Failed after 3 retries| P_Fail["State: Provisioning Failed<br/>Displays friendly preparing screen"]
P_Fail -->|Operator intervention| P_Retry["Retry from failed step<br/>via Provisioning Monitor"]
P_Retry --> P1
end
%% -------------------------------------------------------------
%% PHASE 2: RUNTIME OPERATIONS & IN-LIFE
%% -------------------------------------------------------------
subgraph Phase2["Phase 2: Runtime Operations & In-Life (Lifetime)"]
P4 --> B1(["State: Active<br/>Workspace Ready"])
subgraph RuntimeOps["Day-to-day Operations"]
B1 --> C1["Access via tenant.root-domain"]
C1 --> C2["Host Resolution & State Guard"]
C2 --> C3["Bind Tenant Request Context & DB Pool"]
C3 --> C4["Execute Business APIs CRM / IAM / Storage"]
C4 --> C5["Monitor Storage Quota & User Seats"]
end
subgraph MaintenanceBranch["Maintenance Mode"]
B1 -->|Operator Triggers Restore / Reset DB| M1["Store previous_state<br/>State: Maintenance"]
M1 --> M2["Block tenant traffic<br/>Serve maintenance page"]
M2 --> M3["Control Plane executes DB operation"]
M3 -->|Completed| M4["Restore previous_state"]
M4 --> B1
end
subgraph SuspensionBranch["Suspension Mode"]
B1 -->|Overdue Billing / ToS Violation / Manual| S1["Store reason >= 10 chars<br/>State: Suspended"]
S1 --> S2["Revoke active sessions within 1 min<br/>Reject API keys & inbound webhooks"]
S2 --> S3["Skip scheduled cron jobs<br/>Preserve 100% tenant business data"]
S3 -->|Remediated & Operator Reactivates| S4["Reactivate with reason"]
S4 --> B1
end
end
%% -------------------------------------------------------------
%% PHASE 3: END OF LIFE & DECOMMISSIONING
%% -------------------------------------------------------------
subgraph Phase3["Phase 3: End of Life & Decommissioning"]
B1 -->|Service Termination Request| D1["Schedule Deletion<br/>Owner only - Reason + Exact Subdomain"]
S1 -->|Contract Terminated| D1
P_Fail -->|Abandoned Onboarding Clean-up| D1
D1 --> D2["Store previous_state<br/>State: Pending Deletion"]
D2 --> D3["Notify tenant owner by email<br/>Start 30-day Grace Period"]
D3 -->|Owner cancels deletion within 30 days| D_Cancel["Cancel Deletion<br/>Restore previous_state"]
D_Cancel --> B1
D3 -->|Grace Period expires without cancellation| D4["Execute Hard Teardown"]
subgraph Teardown["Automated Resource Purging"]
D4 --> T1["Drop Tenant PostgreSQL Database"]
T1 --> T2["Delete Storage Files & Backups"]
T2 --> T3["Purge Tenant Users & Session Tokens"]
T3 --> T4["Update State to Deleted<br/>Publish TenantDeleted Event"]
end
T4 --> H1["90-Day Subdomain Hold Period<br/>Prevent Subdomain Hijacking / Phishing"]
H1 --> H2["Retain Registry Record & Audit History<br/>Compliance & Legal Auditing"]
H2 --> H3(["90 Days Expired: Release Subdomain to public pool"])
end
classDef startEnd fill:#1e293b,stroke:#0f172a,stroke-width:2px,color:#f8fafc;
classDef prov fill:#0284c7,stroke:#0369a1,stroke-width:2px,color:#f0f9ff;
classDef active fill:#16a34a,stroke:#15803d,stroke-width:2px,color:#f0fdf4;
classDef warn fill:#d97706,stroke:#b45309,stroke-width:2px,color:#fffbeb;
classDef danger fill:#dc2626,stroke:#b91c1c,stroke-width:2px,color:#fef2f2;
classDef neutral fill:#475569,stroke:#334155,stroke-width:2px,color:#f8fafc;
class A1,H3 startEnd;
class A2,A3,A4,A5,P1,P2,P3,P4,P_Retry prov;
class B1,C1,C2,C3,C4,C5,M4,S4,D_Cancel active;
class P_Fail,M1,M2,M3 warn;
class S1,S2,S3,D1,D2,D3,D4,T1,T2,T3 danger;
class T4,H1,H2 neutral;
Detailed Sequence: Tenant Request Guarding and State Enforcement¶
Every inbound HTTP request, API call, or scheduled background job passes through the Tenancy Guard. Non-Active tenants are strictly isolated and prevented from acquiring database connection pools.
sequenceDiagram
autonumber
actor User as Customer / API Client
participant Proxy as Reverse Proxy (Nginx)
participant Guard as TenantResolutionGuard
participant Registry as TenantRegistryService
participant Context as TenantRequestContext
participant DBPool as TenantConnectionPool
participant App as Business Feature (CRM/IAM)
User->>Proxy: GET https://acme.<root-domain>/api/customers
Proxy->>Guard: Forward request with Host header
Guard->>Registry: Resolve tenant by subdomain "acme"
alt Subdomain does not exist
Registry-->>Guard: null
Guard-->>User: 404 Not Found (Unknown workspace)
else Tenant exists in Registry
Registry-->>Guard: TenantEntity (state, database_name, limits)
alt State == "Provisioning" or "Provisioning Failed"
Guard-->>User: 423 Locked / Redirect (Workspace being prepared)
else State == "Maintenance"
Guard-->>User: 503 Service Unavailable (Maintenance page)
else State == "Suspended"
Guard-->>User: 403 Forbidden (Workspace suspended, contact support)
else State == "Pending Deletion"
Guard-->>User: 410 Gone (Workspace scheduled for deletion)
else State == "Deleted"
Guard-->>User: 404 Not Found (Workspace deleted)
else State == "Active"
Guard->>Context: Bind current tenant context to request
Guard->>DBPool: Get or initialize connection pool for "acme"
DBPool-->>Guard: Active DB Client
Guard->>App: Proceed to Controller & Use Case
App-->>User: 200 OK + Payload
end
end
Requirements¶
| ID | Priority | Requirement |
|---|---|---|
| TLF-01 | Must | The platform shall maintain an explicit 7-state lifecycle: Provisioning, Provisioning failed, Active, Maintenance, Suspended, Pending deletion, and Deleted. |
| TLF-02 | Must | Creation of a tenant starts with state Provisioning and triggers an idempotent 7-step provisioning pipeline. The tenant moves to Active only when all steps succeed. |
| TLF-03 | Must | If provisioning fails after 3 automated backoff retries, state moves to Provisioning failed. An operator can inspect logs and retry from the failed step. |
| TLF-04 | Must | Runtime requests (HTTP, webhooks, cron jobs) shall proceed only when state is Active. All other states reject traffic with distinct error codes and never open a database connection pool. |
| TLF-05 | Must | An operator may suspend an Active tenant with a mandatory reason (>= 10 characters). Within 1 minute, existing user sessions are invalidated, API keys and inbound webhooks are rejected, and crons are skipped. Business data is preserved completely. |
| TLF-06 | Must | Reactivating a Suspended tenant requires a mandatory reason (>= 10 characters). All users, credentials, and features resume immediately without data loss. |
| TLF-07 | Must | Operator maintenance actions (restore, reset) transition the tenant to Maintenance while storing previous_state. Exiting maintenance restores previous_state. |
| TLF-08 | Must | Scheduling deletion requires the Owner role, a mandatory reason (>= 10 characters), and exact typing of the tenant subdomain. The tenant enters Pending deletion for a 30-day grace period. |
| TLF-09 | Must | Within the 30-day grace period, the Owner can cancel deletion, which restores the tenant to its previous_state (Active or Suspended). |
| TLF-10 | Must | When the 30-day grace period expires, hard teardown removes the tenant database, storage files, backups, and user bindings. State becomes Deleted. The registry record is retained for audit history, and the subdomain is held for 90 days before release. |
Screens and Operator Workflows¶
- Customer Sign-Up Wizard (
/sign-up/plan): 4-step wizard (Plan, Company Details, Payment, Done) that registers the tenant row in the control plane. - Provisioning Monitor (
/provisioning): Visual step timeline for each run with elapsed time, error inspection, and "Retry from this step" action. - Tenant Management List (
/tenants): Shows all tenants with badges, health indicators, storage gauges, and action menu (View details, Edit, Suspend/Reactivate, Delete). - Tenant Detail Overview (
/tenants/:id): Displays company details, owner information, storage donut, user roster, backup status, and action buttons conforming to the state matrix. - Lifecycle Action Dialogs: Mandatory confirmation dialogs requiring reasons and subdomain matching for high-impact transitions.
Business rules¶
Allowed Transitions and Role Permissions Matrix¶
| From State | Event / Trigger | Allowed Roles | Target State | Previous State Stored? |
|---|---|---|---|---|
| None | Sign-up wizard submit | System / Guest | Provisioning |
No |
Provisioning |
Pipeline succeeds | System | Active |
No |
Provisioning |
Pipeline fails (3 retries) | System | Provisioning failed |
No |
Provisioning failed |
Retry succeeds | Operator / Owner | Active |
No |
Active |
Suspend | Operator / Owner | Suspended |
Yes |
Suspended |
Reactivate | Operator / Owner | Active |
No |
Active |
Restore / Reset DB | Operator / Owner | Maintenance |
Yes (Active) |
Suspended |
Restore / Reset DB | Operator / Owner | Maintenance |
Yes (Suspended) |
Maintenance |
Maintenance completed | System / Operator | Restore to previous | No |
Active |
Schedule deletion | Owner | Pending deletion |
Yes (Active) |
Suspended |
Schedule deletion | Owner | Pending deletion |
Yes (Suspended) |
Provisioning failed |
Schedule deletion | Owner | Pending deletion |
Yes (Provisioning failed) |
Pending deletion |
Cancel deletion | Owner | Restore to previous | No |
Pending deletion |
30-day grace period ends | System / Cron | Deleted |
No |
Subdomain Protection and Safety Invariants¶
- Reserved Subdomains:
admin,api,app,www,status, andmailare permanently reserved and rejected regardless of letter casing. - Subdomain Hold Period: When a tenant is hard-deleted, its subdomain remains blocked for a default of 90 days to prevent domain hijacking, phishing, or accidental message interception.
- Audit Trail Immutability: Tenant deletion purges tenant-scoped data and user records, but leaves the platform control-plane registry entry and audit logs intact with state
Deleted.
Acceptance criteria¶
- TLF-01: The system defines and strictly enforces all 7 lifecycle states without allowing undefined intermediate states.
- TLF-02: A successful sign-up creates a
Provisioningrecord and triggers the 7-step provisioning flow to completion, ending inActive. - TLF-03: When step 2 (Create database) fails, retrying twice and succeeding marks the run as successful; failing 3 times transitions the tenant to
Provisioning failed. - TLF-04: Sending an HTTP request to
foo.<root-domain>whenfoois inSuspendedorMaintenancereturns a specific 403 or 503 error code without opening a PostgreSQL connection pool. - TLF-05: Suspending an active tenant terminates existing JWT/cookie sessions within 1 minute and stops scheduled cron tasks.
- TLF-06: Reactivating a suspended tenant with reason restores user access without altering tenant data.
- TLF-07: Putting a suspended tenant into maintenance for database restore returns the tenant to
Suspendedupon completion. - TLF-08: Attempting to schedule deletion with a mismatched subdomain input fails and maintains the current state.
- TLF-09: Cancelling deletion on a tenant scheduled from
Suspendedreturns it toSuspended. - TLF-10: After the 30-day grace period, background worker drops the tenant database and files, updates the state to
Deleted, and blocks registration of the subdomain for 90 days.
Out of scope¶
- Automated domain name DNS purchases for custom white-label customer domains (e.g.
erp.customer.com). - Manual tenant creation bypass without going through the control-plane provisioning pipeline.
- Cross-tenant data merging or splitting.