Testland
Browse all skills & agents

tenant-onboarding-test-author

Workflow-driven skill that authors a test suite for tenant provisioning and offboarding: account creation, isolation at creation (no cross-tenant bleed from a new tenant's first API call), default resource quotas, billing record linkage, seed and default data correctness, idempotent re-provisioning, and teardown with full data deletion. Walks through mapping provisioning surfaces, generating test cases per surface, emitting the test suite skeleton (pytest / Jest / JUnit / Go test), and producing a coverage matrix. Use when a new tenant onboarding flow is introduced or changed, when the offboarding pipeline is modified, or when auditing provisioning coverage before a compliance review. Distinct from tenant-leak-test-author (runtime cross-tenant access) and cross-tenant-data-leak-tests (CI gate): this skill covers the provisioning lifecycle, not steady-state access control.

Install with skills.sh (any agent)

npx skills add testland/qa --skill tenant-onboarding-test-author
View source

tenant-onboarding-test-author

Overview

Tenant onboarding and offboarding are lifecycle transitions, not steady-state access patterns. A new tenant's provisioning pipeline touches account records, isolation boundaries, quota tables, billing systems, seed data loaders, and delete-cascade paths - surfaces the runtime isolation test battery never exercises.

This skill produces a test suite for that lifecycle. The output is committed to the project repo alongside the runtime suite from tenant-leak-test-author.

The workflow is:

  1. Map the provisioning pipeline surfaces.
  2. Enumerate test scenarios per surface.
  3. Generate test case stubs (fixtures + assertions).
  4. Emit the test suite skeleton.
  5. Build the coverage matrix.

Differentiation

SkillWhat it tests
tenant-onboarding-test-author (this skill)Provisioning lifecycle: creation, quota, billing, seed data, idempotency, teardown
tenant-leak-test-authorSteady-state runtime: cross-tenant access attempts, IDOR, horizontal escalation
cross-tenant-data-leak-testsCI gate: confirms isolation invariants pass before merge

How to use

  1. Trace the onboarding and offboarding code path and inventory every state-changing surface, using references/provisioning-surface-map.md.
  2. For each surface, pull the scenarios that must hold from references/provisioning-test-scenarios.md - creation, isolation-at-create, quota, billing, seed, idempotency, teardown.
  3. Name each test test_<surface>_<scenario>_<expected>() and declare the shared fixtures (Step 3 table).
  4. Pick the framework for your stack (Step 4 table) and emit the suite skeleton from references/test-skeleton-examples.md.
  5. Build the coverage matrix (Step 5); treat every empty cell as a gap to justify or schedule.
  6. Run the suite as a non-privileged app role (not a superuser) so offboarding and RLS assertions stay valid, then cross-check the Anti-patterns table.

Step 1 - Map provisioning pipeline surfaces

Walk the onboarding code path end-to-end and enumerate every surface that changes state during provisioning or deprovisioning - tenant registry, identity store, database layer, object storage, quota table, billing record, seed data, re-provisioning path, and teardown. The full surface inventory table, plus the AWS Well-Architected SaaS Lens and Microsoft lifecycle gating notes: references/provisioning-surface-map.md.

Step 2 - Enumerate test scenarios per surface

For each surface from Step 1, identify the scenarios that must hold: account creation atomicity, isolation from the first call, default quota values, billing linkage, seed-data scoping, idempotent re-provisioning, and teardown with full deletion after the retention period. The full per-surface scenario catalog with its Microsoft citations: references/provisioning-test-scenarios.md.

Step 3 - Generate test cases

Naming convention matches tenant-leak-test-author:

test_<surface>_<scenario>_<expected>()

Examples:

test_account_creation_duplicate_slug_returns_conflict()
test_isolation_new_tenant_invisible_to_existing_tenant_immediately()
test_quota_default_values_match_tier_config_on_creation()
test_billing_record_exists_and_references_tenant_id_on_creation()
test_seed_data_scoped_to_new_tenant_only()
test_provisioning_idempotent_no_duplicate_quota_row()
test_offboarding_application_data_deleted_after_retention_period()

Required fixtures

FixturePurpose
existing_tenantA fully-provisioned tenant in steady state (pre-exists the new tenant)
new_tenantThe tenant being provisioned under test
new_tenant_ownerThe user who triggered provisioning
pricing_tierThe tier config record that defines default quotas and features
billing_stubA test double for the payment provider that records linkage calls
offboarded_tenantA tenant that has been offboarded; used for teardown assertions

Step 4 - Pick the test framework and emit skeleton

Same framework selection table as tenant-leak-test-author:

StackFramework
Python (Django/Flask/FastAPI)pytest
Node (Express/Nest)Jest or Vitest + Supertest
JVM (Spring/Quarkus)JUnit 5 + Testcontainers
Gotesting + httptest
Ruby (Rails)RSpec + request specs

The full pytest skeleton (one test per Step 2 scenario) and a language-agnostic SQL idempotency assertion: references/test-skeleton-examples.md.

Step 5 - Coverage matrix

Track one cell per (surface, scenario). Empty cells are coverage gaps that the PR author must justify or schedule.

Surface                   | create | isolation_at_create | quota | billing | seed | idempotent | teardown
account_record            |   X    |                     |       |         |      |     X      |    X
identity_bindings         |   X    |                     |       |         |      |     X      |    X
database_isolation        |        |          X          |       |         |      |            |    X
quota_table               |        |                     |   X   |         |      |     X      |    X
billing_record            |        |                     |       |    X    |      |     X      |    X
seed_data                 |        |          X          |       |         |  X   |            |    X
feature_flags             |        |                     |   X   |         |  X   |            |

Generate this matrix from Step 1's surface inventory plus Step 2's scenario list. Populate it cell-by-cell as test cases are written.

Anti-patterns

Anti-patternWhy it failsFix
Testing only happy-path provisioningMisses atomicity on mid-flow failure; offboarding is never exercisedAdd failure-injection and teardown cases
Asserting isolation only in steady statePer Microsoft tenancy models guidance, isolation must hold from the first callAdd immediate post-creation cross-tenant probe
Skipping idempotency testsRe-provisioning retries after transient failures; duplicate records cause silent billing or quota bugsAdd second-call and partial-failure-retry cases
Hardcoding expected quota values as literalsQuota values change with pricing tiers; tests break on plan changesRead expected values from the tier config fixture
Deleting test tenants without asserting deletionTeardown path is untested; data retention bugs go undetectedAssert row counts = 0 across all tenant-bearing tables after delete
Using a superuser connection for post-offboarding assertionsSuperuser bypasses RLS and soft-delete filters; tests pass even when app-layer deletion is brokenUse the app-role or the API layer to verify absence
Treating provisioning as atomic when it is notCloud provisioning pipelines are often eventually consistent; test may pass before side-effects completeUse explicit wait/poll or event-driven fixtures that confirm each step before asserting

Worked example

A Django SaaS adds a Team-tier onboarding flow. Step 1 inventories the surfaces it touches: the tenants registry, identity role bindings, a TenantQuota row, a billing_subscriptions record, and a seeded welcome project. Step 2 selects the scenarios: duplicate slug returns conflict, the new tenant is invisible to an existing tenant immediately, the quota matches the tier default, billing is linked, a second provisioning call stays idempotent, and offboarding deletes the data.

For idempotency, Step 3 names the test test_provisioning_idempotent_no_duplicate_quota_row() and Step 4 emits the pytest body from the skeleton: it POSTs /api/tenants/ a second time with the same slug, then asserts TenantQuota.objects.filter(tenant=new_tenant).count() == 1. Run as the non-superuser app role, the second call reuses the existing record rather than inserting a duplicate, so the assertion passes. The quota_table x idempotent cell in the Step 5 matrix flips to X.

Output

This skill produces:

  • A provisioning surface inventory (Step 1).
  • A scenario list per surface (Step 2).
  • A test suite skeleton (Steps 3-4) committed to the project repo.
  • A coverage matrix (Step 5).

The runtime isolation gate is cross-tenant-data-leak-tests.

References

Provisioning pipeline surface map

View source (opens in new window)

Provisioning pipeline surface map

Walk the onboarding code path end-to-end and enumerate every surface that changes state during provisioning or deprovisioning. Per the AWS Well-Architected SaaS Lens (docs.aws.amazon.com/wellarchitected/latest/saas-lens/tenant-isolation.html (opens in new window)), isolation must be established at every layer of the stack independently:

SurfaceWhat to captureHow to find it
Tenant registry / catalogRecord created, unique key assignedCentral tenants table or tenant-management service
Identity storeUser account created, role bindingsAuth provider admin API or IAM config
Database layerSchema/row/database provisioned per isolation modelORM migration runner, schema-per-tenant scripts
Object storageBucket or prefix created with scoped policyIaC for storage resources
Quota / rate-limit tableDefault quotas inserted for the new tenantQuota service or DB seeding script
Billing recordBilling entry linked to tenant IDBilling service or payment-provider webhook handler
Seed / default dataFeature flags, default roles, template data loadedSeed scripts, fixture loaders
Re-provisioning pathIdempotent re-run: no duplicate records, no failed constraintsProvisioning entry point called twice
Offboarding / teardownData deletion, billing cancellation, quota cleanupDelete or deactivate endpoint

Per the Microsoft Multitenant Architecture Center (learn.microsoft.com/en-us/azure/architecture/guide/multitenant/considerations/tenant-life-cycle (opens in new window)), onboarding must account for data residency, compliance tier, billing model, and disaster-recovery SLOs, each of which can gate provisioning steps. Capture which conditions gate each surface - a compliance-gated provisioning step needs its own test branch.

Per the Microsoft resource organisation guidance (learn.microsoft.com/en-us/azure/architecture/guide/multitenant/approaches/resource-organization (opens in new window)), quota and subscription limits must be modelled per tenant at provisioning time. Record the expected default quota values - they are assertions, not implementation details.

Provisioning test scenarios per surface

View source (opens in new window)

Provisioning test scenarios per surface

For each surface from the surface map, identify the scenarios that must hold.

Account creation

  • Tenant record persists with a non-null unique identifier after a successful provisioning call.
  • Provisioning a second tenant with the same email or slug returns an error (no silent duplication).
  • A provisioning call that fails mid-flow leaves no partial record (atomicity).

Isolation at creation (no cross-tenant bleed)

Per the Microsoft tenancy models guidance (learn.microsoft.com/en-us/azure/architecture/guide/multitenant/considerations/tenancy-models (opens in new window)), isolation must hold from the first request a tenant makes, not only after steady-state data accumulates. Test cases:

  • Immediately after Tenant B is provisioned, a call authenticated as Tenant A cannot enumerate Tenant B's resources (list endpoint returns 0 items for A).
  • Tenant B's provisioned schema or row-level security policy is in place before the first application-level write is accepted.
  • No existing tenant's data becomes visible to the new tenant through shared infrastructure (cache warm-up, search index bootstrap, etc.).

Default resource quotas

Per the Microsoft consumption measurement guidance (learn.microsoft.com/en-us/azure/architecture/guide/multitenant/considerations/measure-consumption (opens in new window)), per-tenant consumption limits should be tracked from provisioning. Test cases:

  • A new tenant's quota record exists immediately after provisioning and matches the expected default values for the tenant's pricing tier.
  • Attempting an action that exceeds the default quota (e.g., creating one more resource than the limit allows) returns a quota-exceeded error, not a generic error or a silent failure.
  • Upgrading a tenant's tier updates the quota record (verify the seeded default and the upgrade delta separately).

Billing record linkage

Per the Microsoft tenant lifecycle guidance (learn.microsoft.com/en-us/azure/architecture/guide/multitenant/considerations/tenant-life-cycle (opens in new window)), billing linkage is a first-class onboarding concern. Test cases:

  • A billing record (subscription, customer ID, or metering entry) exists and references the tenant's canonical identifier within the same provisioning transaction.
  • A provisioning call with an invalid payment method does not create a half-provisioned tenant.
  • The billing record's tier matches the plan chosen at signup.

Seed and default data

  • Feature flags for the tenant's tier are present with correct default values.
  • Default roles (e.g., owner, member, viewer) exist and are assigned to the provisioning user.
  • Template or onboarding data (welcome project, sample records) is scoped exclusively to the new tenant - it does not appear in any other tenant's list endpoints.

Idempotent re-provisioning

An idempotent provisioning call is one that produces the same final state regardless of how many times it is invoked. Test cases:

  • Calling the provisioning endpoint a second time with the same input returns a success response (or an explicit "already exists" response) without creating duplicate records.
  • All constraint-unique fields (quota row, billing record, identity binding) remain singular after two provisioning calls.
  • A re-provisioning call after a partial failure completes successfully without manual cleanup.

Teardown and offboarding

Per the Microsoft tenant lifecycle guidance (learn.microsoft.com/en-us/azure/architecture/guide/multitenant/considerations/tenant-life-cycle (opens in new window)), offboarding must define a retention period and support re-onboarding during that window. Test cases:

  • After offboarding, the tenant's application data (records, files, jobs) is not accessible through any API endpoint.
  • After offboarding, no row for the tenant exists in quota, billing, or identity tables (or rows are flagged deleted and excluded from active queries).
  • Re-onboarding the same tenant during the retention period succeeds without data loss if re-onboarding is a supported operation.
  • After the retention period expires (simulate with a fixed past timestamp), a hard-delete job removes all remaining rows; assert count = 0 across all tenant-bearing tables.

Provisioning test suite skeleton examples

View source (opens in new window)

Provisioning test suite skeleton examples

Concrete skeletons for the emitted suite. Pick the framework from the selection table in SKILL.md; the pytest example below maps one-to-one to the scenarios in provisioning-test-scenarios.md (opens in new window).

Example skeleton (pytest)

import pytest

class TestTenantProvisioning:
    """Provisioning lifecycle tests - distinct from runtime isolation battery."""

    def test_account_creation_stores_tenant_record(
        self, provisioning_client, pricing_tier
    ):
        response = provisioning_client.post(
            "/api/tenants/",
            data={"slug": "acme", "plan": pricing_tier.id, "owner_email": "owner@acme.com"}
        )
        assert response.status_code == 201
        assert response.json()["id"] is not None

    def test_isolation_new_tenant_invisible_to_existing_tenant_immediately(
        self, client, existing_tenant_user, new_tenant
    ):
        # New tenant was just provisioned; existing tenant must not see it
        client.force_login(existing_tenant_user)
        response = client.get("/api/workspaces/")
        ids = {w["id"] for w in response.json()["results"]}
        assert new_tenant.id not in ids

    def test_quota_default_matches_tier_on_creation(
        self, new_tenant, pricing_tier
    ):
        from quotas.models import TenantQuota
        quota = TenantQuota.objects.get(tenant=new_tenant)
        assert quota.max_users == pricing_tier.default_max_users
        assert quota.max_storage_gb == pricing_tier.default_max_storage_gb

    def test_billing_record_linked_on_creation(
        self, new_tenant, billing_stub
    ):
        assert billing_stub.was_called_for(new_tenant.id), (
            "Billing provider must be notified during provisioning"
        )
        record = billing_stub.get_record(new_tenant.id)
        assert record["plan"] == new_tenant.plan

    def test_seed_data_scoped_to_new_tenant_only(
        self, client, existing_tenant_user, new_tenant
    ):
        # Existing tenant must not see new tenant's seed data in shared endpoints
        client.force_login(existing_tenant_user)
        response = client.get("/api/projects/")
        tenant_ids = {p["tenant_id"] for p in response.json()["results"]}
        assert str(new_tenant.id) not in tenant_ids

    def test_provisioning_idempotent_no_duplicate_quota_row(
        self, provisioning_client, new_tenant, pricing_tier
    ):
        # Call provisioning a second time with the same input
        provisioning_client.post(
            "/api/tenants/",
            data={"slug": new_tenant.slug, "plan": pricing_tier.id,
                  "owner_email": "owner@acme.com"}
        )
        from quotas.models import TenantQuota
        count = TenantQuota.objects.filter(tenant=new_tenant).count()
        assert count == 1, "Idempotent re-provisioning must not create duplicate quota rows"

    def test_offboarding_deletes_application_data(
        self, admin_client, offboarded_tenant
    ):
        admin_client.delete(f"/api/tenants/{offboarded_tenant.id}/")
        response = admin_client.get(f"/api/tenants/{offboarded_tenant.id}/projects/")
        assert response.status_code == 404

Example: idempotency via SQL assertion (language-agnostic)

-- After two provisioning calls for the same tenant slug:
SELECT count(*) FROM tenant_quotas WHERE tenant_id = '<new_tenant_uuid>';
-- Expected: 1 (not 2)

SELECT count(*) FROM billing_subscriptions WHERE tenant_id = '<new_tenant_uuid>';
-- Expected: 1 (not 2)

Related skills

cross-tenant-data-leak-tests

Workflow-driven skill that emits the runtime CI gate of cross-tenant leak tests - the actual battery a multi-tenant codebase must pass on every PR. Defines the canonical test patterns (read-other-tenant-by-id, list-leak, spoofed-tenant-id-in-body, JWT-replay, FK-cross-tenant, unique-collision side channel, object-storage IDOR, search-index-direct-query, async-job-context-reload, cache-key-collision), the expected response codes per pattern (404 vs 403 disclosure trade-off), the Postgres-RLS-direct test patterns, and the CI integration (run with non-superuser non-BYPASSRLS role, fail the build on any leak). Use when implementing the actual leak-test suite (after tenant-leak-test-author produces the plan), when adding the CI gate to an existing project, or when investigating a leak finding.

multi-engine-row-level-security-reference

Pure-reference catalog of row/tenant isolation mechanisms across four database engines: MySQL and MariaDB (no native RLS - views with SQL SECURITY INVOKER plus app-layer enforcement), CockroachDB (native RLS via ALTER TABLE ENABLE ROW LEVEL SECURITY and CREATE POLICY, matching Postgres semantics), Vitess (keyspace sharding + vindexes route tenant writes to dedicated shards without a policy layer), and SQL Server (CREATE SECURITY POLICY with inline table-valued function filter/block predicates). Covers the isolation mechanism, tenant-context pattern, bypass risks, and test patterns for each engine. Use when designing or auditing tenant isolation on MySQL, MariaDB, CockroachDB, Vitess, or SQL Server.

row-level-security-postgres-reference

Pure-reference catalog of Postgres Row-Level Security (RLS) for tenant isolation. Covers enabling RLS (ALTER TABLE ... ENABLE ROW LEVEL SECURITY, default-deny semantics), CREATE POLICY syntax (USING vs WITH CHECK clauses, FOR SELECT/INSERT/UPDATE/DELETE/ALL, permissive vs restrictive, TO role_name), bypassing RLS (superuser / BYPASSRLS / table owner / FORCE ROW LEVEL SECURITY), tenant context patterns (current_user, current_setting, JWT claims via Supabase auth.uid() / auth.jwt()), performance discipline (wrapping auth functions in SELECT, index on policy-referenced columns), and anti-patterns. Use as the RLS-pattern reference for Postgres-backed tenant isolation. Consumed by tenant-leak-test-author, cross-tenant-data-leak-tests.

tenant-isolation-models-reference

Pure-reference catalog of tenant-isolation models for B2B SaaS. Defines the isolation continuum from full-isolation (separate compute + data + network per tenant) to fully-shared (one deployment, tenant_id discriminator), names the canonical models (Microsoft's automated-single-tenant / fully-multitenant / vertically-partitioned / horizontally-partitioned; AWS Well-Architected's silo / pool / bridge framing; deployment-stamps / supertenants terminology), enumerates the trade-offs (cost, blast radius, noisy neighbor, compliance, scale limits), and lists the test surfaces each model creates (cross-tenant data leak, tenant-id propagation, deployment-routing). Use as the model-selection reference when designing or auditing tenant isolation. Consumed by tenant-leak-test-author, cross-tenant-data-leak-tests.

tenant-leak-test-author

Workflow-driven skill that builds a tenant-leak test plan from an inventory of tenant-bearing surfaces (database tables, APIs, object storage, search indices, async messages) and the isolation model in use. Walks through identifying tenant-bearing surfaces, enumerating the attack patterns per OWASP WSTG-ATHZ-02 (horizontal escalation, vertical escalation, IDOR / BOLA), generating test cases that exercise each pattern against each surface, and emitting the test suite skeleton (pytest / Jest / JUnit / Go test) with explicit cross-tenant probes. Use when designing a multi-tenant test suite for a new feature, when auditing test coverage for an existing tenant boundary, or when reviewing PRs that add tenant-bearing surfaces. Distinct from cross-tenant-data-leak-tests which is the runtime gate; this skill produces the plan.