cross-tenant-data-leak-tests
Workflow-driven skill that emits the runtime CI gate of cross-tenant leak tests - the actual battery a multi-tenant codebase must pass on every PR. Defines the canonical test patterns (read-other-tenant-by-id, list-leak, spoofed-tenant-id-in-body, JWT-replay, FK-cross-tenant, unique-collision side channel, object-storage IDOR, search-index-direct-query, async-job-context-reload, cache-key-collision), the expected response codes per pattern (404 vs 403 disclosure trade-off), the Postgres-RLS-direct test patterns, and the CI integration (run with non-superuser non-BYPASSRLS role, fail the build on any leak). Use when implementing the actual leak-test suite (after tenant-leak-test-author produces the plan), when adding the CI gate to an existing project, or when investigating a leak finding.
Install with skills.sh (any agent)
npx skills add testland/qa --skill cross-tenant-data-leak-testscross-tenant-data-leak-tests
Overview
The runtime CI gate. While tenant-leak-test-author produces the plan (which surfaces, which patterns), this skill produces the executing tests - the actual code that fails the build when isolation breaks.
The contract:
When to use
Step 1 - Choose the test runner and DB connection role
For Postgres-backed apps, this is the most-common-bug step:
| Connection used in test | Result |
|---|---|
| Superuser | Bypasses RLS - tests pass, prod leaks. Do not use. |
Role with BYPASSRLS attribute | Same as superuser. Do not use. |
Role that owns the tenant table (and table not FORCEd) | Bypasses RLS. Do not use unless FORCE ROW LEVEL SECURITY is set. |
| Plain application role (no BYPASSRLS, not owner) | Correct - same role prod uses. |
Per row-level-security-postgres-reference, verify with:
SELECT rolname, rolsuper, rolbypassrls FROM pg_roles
WHERE rolname = current_user;
-- Expect: rolsuper=f, rolbypassrls=fFor Django, set DATABASES['default']['USER'] to the app role in the test settings. For Rails, config/database.yml test section. For Spring Boot, spring.datasource.username in application-test.yml.
Step 2 - The canonical battery
These tests should exist for every tenant-bearing API surface.
Test 1 - Read-other-tenant-by-id
def test_get_other_tenant_resource_returns_404(
self, client, tenant_a_user, tenant_b_resource
):
client.force_login(tenant_a_user)
resp = client.get(f"/api/documents/{tenant_b_resource.id}/")
assert resp.status_code == 404 # not 403 - avoid existence disclosureConvention: return 404, not 403, for resources the requester can't access in another tenant. 403 leaks existence (tenant A learns tenant B has resource with this ID). The trade-off: debugging slightly harder. Document the project's choice.
Test 2 - List-leak
def test_list_does_not_include_other_tenant_resources(
self, client, tenant_a_user, tenant_b_resource
):
client.force_login(tenant_a_user)
resp = client.get("/api/documents/")
assert resp.status_code == 200
ids = {d["id"] for d in resp.json()["results"]}
assert tenant_b_resource.id not in idsEven with RLS enforcing visibility, application-layer caches can leak. Test against fresh queries.
Test 3 - Spoofed-tenant-id-in-body
def test_tenant_id_in_body_ignored_or_rejected(
self, client, tenant_a_user, tenant_b
):
client.force_login(tenant_a_user)
resp = client.post("/api/documents/", json={
"tenant_id": str(tenant_b.id),
"body": "leak"
})
if resp.status_code == 201:
created = resp.json()
assert created["tenant_id"] != str(tenant_b.id), \
"Server accepted tenant_id from body - must derive from session"Test 4 - JWT replay across tenants
def test_jwt_for_tenant_a_rejected_on_tenant_b_path(
self, client, tenant_a_user, tenant_b_resource
):
token = sign_jwt(tenant_a_user)
resp = client.get(
f"/api/documents/{tenant_b_resource.id}/",
HTTP_AUTHORIZATION=f"Bearer {token}"
)
assert resp.status_code in (401, 404)If the API has tenant-scoped paths (/api/tenants/<id>/...), also test that tenant A's JWT cannot be used with tenant B's path even with a valid signature - the tenant_id claim must be checked against the path.
Test 5 - FK cross-tenant
def test_cannot_create_fk_referencing_other_tenant(
self, client, tenant_a_user, tenant_b_resource
):
client.force_login(tenant_a_user)
# tenant_b_resource exists, but A shouldn't reference it
resp = client.post("/api/comments/", json={
"document_id": str(tenant_b_resource.id),
"body": "comment on other tenant's doc"
})
assert resp.status_code in (400, 404)This tests FK-based leak via reference: the FK constraint bypasses RLS per row-level-security-postgres-reference, so the FK must be validated at application layer too.
Test 6 - Unique-collision side channel
def test_unique_violation_does_not_disclose_other_tenant_existence(
self, client, tenant_a_user, tenant_b_resource_with_slug
):
client.force_login(tenant_a_user)
# Try to create with same slug as tenant B
resp = client.post("/api/documents/", json={
"slug": tenant_b_resource_with_slug.slug,
"body": "x"
})
# Should succeed (RLS scopes the unique check to tenant A)
# or fail with a non-disclosing error if unique is global
if resp.status_code in (409, 422):
assert "tenant_b" not in resp.text.lower()
assert tenant_b_resource_with_slug.id not in resp.textPer row-level-security-postgres-reference: "Foreign key constraint checks, Unique constraint checks, TRUNCATE, and REFERENCES privilege checks bypass RLS." Solution: make slug unique per tenant: UNIQUE (tenant_id, slug).
Test 7 - Object-storage IDOR
def test_storage_presigned_url_path_traversal_denied(
self, client, tenant_a_user, tenant_b_resource
):
client.force_login(tenant_a_user)
presigned = client.get(
f"/api/documents/{tenant_a_user.tenant_id}/file/"
).json()["url"]
# Modify the URL to point at tenant B's prefix
leaked_url = presigned.replace(
str(tenant_a_user.tenant_id),
str(tenant_b_resource.tenant_id)
)
resp = requests.get(leaked_url)
assert resp.status_code == 403The S3 / GCS bucket policy must enforce the prefix - application code is not sufficient.
Test 8 - Search-index direct query
def test_search_query_must_include_tenant_filter(
self, opensearch_client, tenant_a, tenant_b_resource
):
# Direct ES query without tenant filter (simulating leaked path)
result = opensearch_client.search(
index="documents",
body={"query": {"match_all": {}}}
)
# Test asserts the API endpoint always adds a tenant filter.
# The DIRECT search above should not be reachable from any API path.
# This test ensures no route exists that issues unfiltered search.
for route in app.url_map.iter_rules():
if "search" in route.endpoint:
assert "@tenant_required" in inspect.getsource(
app.view_functions[route.endpoint]
)Test 9 - Async-job context-reload
def test_async_job_reloads_tenant_from_db_not_payload(
self, tenant_a, tenant_b_resource
):
# Enqueue a job with a payload pointing at tenant B's resource
job = enqueue_export(
resource_id=tenant_b_resource.id,
# Crafted to spoof - but executor must verify
tenant_id_claim=tenant_a.id
)
result = run_job(job)
# Executor must derive tenant_id from resource_id, not the payload
assert result.tenant_id == tenant_b_resource.tenant_idTest 10 - Cache key collision
def test_cache_keys_are_tenant_scoped(self, cache, tenant_a, tenant_b):
cache.set("user:1", "tenant_a_data", tenant_id=tenant_a.id)
cache.set("user:1", "tenant_b_data", tenant_id=tenant_b.id)
assert cache.get("user:1", tenant_id=tenant_a.id) == "tenant_a_data"
assert cache.get("user:1", tenant_id=tenant_b.id) == "tenant_b_data"Cache wrappers must prepend tenant_id to every key.
Postgres-RLS-direct (test 1 - 6 at DB layer)
Run these as the application role (not superuser):
BEGIN;
SET LOCAL ROLE app_user;
SET LOCAL app.tenant_id = '<tenant_a_uuid>';
INSERT INTO documents (id, tenant_id, body)
VALUES (gen_random_uuid(), current_setting('app.tenant_id')::uuid, 'a');
SET LOCAL app.tenant_id = '<tenant_b_uuid>';
SELECT count(*) FROM documents; -- expect 0
-- Cross-tenant INSERT
INSERT INTO documents (tenant_id, body) VALUES ('<tenant_a_uuid>', 'leak');
-- Expect: ERROR: new row violates row-level security policy for table "documents"
ROLLBACK;Per row-level-security-postgres-reference: test fails if either assertion fails (count != 0, or INSERT succeeds).
Step 3 - CI integration
# .github/workflows/tenant-isolation.yml
name: tenant-isolation
on:
pull_request:
paths:
- "**/*.py"
- "**/migrations/**"
- ".github/workflows/tenant-isolation.yml"
jobs:
cross-tenant-tests:
runs-on: ubuntu-latest
services:
postgres:
image: postgres:16
env:
POSTGRES_PASSWORD: postgres
options: >-
--health-cmd pg_isready
--health-interval 10s
ports:
- 5432:5432
steps:
- uses: actions/checkout@v5
- uses: actions/setup-python@v5
with:
python-version: '3.12'
- name: Install
run: pip install -e ".[test]"
- name: Create non-superuser role
env:
PGPASSWORD: postgres
run: |
psql -h localhost -U postgres -d postgres -c "
CREATE ROLE app_user LOGIN PASSWORD 'app';
"
- name: Apply migrations
env:
DATABASE_URL: postgresql://postgres:postgres@localhost/test
run: python manage.py migrate
- name: Run cross-tenant suite
env:
DATABASE_URL: postgresql://app_user:app@localhost/test
run: pytest tests/tenant_isolation/ --tb=short --no-header -vKey: the test job connects as app_user, not as the postgres superuser. The migrations run as superuser; the tests run as the application role.
Step 4 - Diagnose a failure
When a leak test fails:
Anti-patterns
| Anti-pattern | Why it fails | Fix |
|---|---|---|
| Tests connect as superuser | RLS bypassed; tests pass falsely | Use prod-equivalent role |
| Migrations and tests use same role | Migrations need DDL; tests should not | Two roles: migrations user (owner), app user (RLS-bound) |
| Test setup creates tenants via direct SQL bypassing RLS | OK in setup, but reset tenant context before assertions | SET LOCAL between fixture creation and test body |
| 403 used instead of 404 for unauthorised cross-tenant resources | Leaks existence | Use 404 (and document) |
| Single test for "tenant isolation works" | Insufficient coverage | One test per (surface, pattern) cell |
| Skipping FK/UNIQUE side-channel tests | Real production bugs hide here | Always test FK + unique cross-tenant |
| No async-job context-reload test | Job runners often trust payload | Always test |
| Cache without tenant prefix | Tenant A and B alias same logical key | Verify key generation in test |
| Suite not part of CI | Catches nothing | Block merge on failure |
Limitations
References
Related skills
multi-engine-row-level-security-reference
Pure-reference catalog of row/tenant isolation mechanisms across four database engines: MySQL and MariaDB (no native RLS - views with SQL SECURITY INVOKER plus app-layer enforcement), CockroachDB (native RLS via ALTER TABLE ENABLE ROW LEVEL SECURITY and CREATE POLICY, matching Postgres semantics), Vitess (keyspace sharding + vindexes route tenant writes to dedicated shards without a policy layer), and SQL Server (CREATE SECURITY POLICY with inline table-valued function filter/block predicates). Covers the isolation mechanism, tenant-context pattern, bypass risks, and test patterns for each engine. Use when designing or auditing tenant isolation on MySQL, MariaDB, CockroachDB, Vitess, or SQL Server.
row-level-security-postgres-reference
Pure-reference catalog of Postgres Row-Level Security (RLS) for tenant isolation. Covers enabling RLS (ALTER TABLE ... ENABLE ROW LEVEL SECURITY, default-deny semantics), CREATE POLICY syntax (USING vs WITH CHECK clauses, FOR SELECT/INSERT/UPDATE/DELETE/ALL, permissive vs restrictive, TO role_name), bypassing RLS (superuser / BYPASSRLS / table owner / FORCE ROW LEVEL SECURITY), tenant context patterns (current_user, current_setting, JWT claims via Supabase auth.uid() / auth.jwt()), performance discipline (wrapping auth functions in SELECT, index on policy-referenced columns), and anti-patterns. Use as the RLS-pattern reference for Postgres-backed tenant isolation. Consumed by tenant-leak-test-author, cross-tenant-data-leak-tests.
tenant-isolation-models-reference
Pure-reference catalog of tenant-isolation models for B2B SaaS. Defines the isolation continuum from full-isolation (separate compute + data + network per tenant) to fully-shared (one deployment, tenant_id discriminator), names the canonical models (Microsoft's automated-single-tenant / fully-multitenant / vertically-partitioned / horizontally-partitioned; AWS Well-Architected's silo / pool / bridge framing; deployment-stamps / supertenants terminology), enumerates the trade-offs (cost, blast radius, noisy neighbor, compliance, scale limits), and lists the test surfaces each model creates (cross-tenant data leak, tenant-id propagation, deployment-routing). Use as the model-selection reference when designing or auditing tenant isolation. Consumed by tenant-leak-test-author, cross-tenant-data-leak-tests.
tenant-leak-test-author
Workflow-driven skill that builds a tenant-leak test plan from an inventory of tenant-bearing surfaces (database tables, APIs, object storage, search indices, async messages) and the isolation model in use. Walks through identifying tenant-bearing surfaces, enumerating the attack patterns per OWASP WSTG-ATHZ-02 (horizontal escalation, vertical escalation, IDOR / BOLA), generating test cases that exercise each pattern against each surface, and emitting the test suite skeleton (pytest / Jest / JUnit / Go test) with explicit cross-tenant probes. Use when designing a multi-tenant test suite for a new feature, when auditing test coverage for an existing tenant boundary, or when reviewing PRs that add tenant-bearing surfaces. Distinct from cross-tenant-data-leak-tests which is the runtime gate; this skill produces the plan.
tenant-onboarding-test-author
Workflow-driven skill that authors a test suite for tenant provisioning and offboarding: account creation, isolation at creation (no cross-tenant bleed from a new tenant's first API call), default resource quotas, billing record linkage, seed and default data correctness, idempotent re-provisioning, and teardown with full data deletion. Walks through mapping provisioning surfaces, generating test cases per surface, emitting the test suite skeleton (pytest / Jest / JUnit / Go test), and producing a coverage matrix. Use when a new tenant onboarding flow is introduced or changed, when the offboarding pipeline is modified, or when auditing provisioning coverage before a compliance review. Distinct from tenant-leak-test-author (runtime cross-tenant access) and cross-tenant-data-leak-tests (CI gate): this skill covers the provisioning lifecycle, not steady-state access control.