Testland
Browse all skills & agents

exploratory-testing

Plans and runs time-boxed exploratory testing when tester hours are scarce before a release - one tester with two free 45-minute blocks before code freeze, a high-stakes window such as year-end payroll, or a device and environment the scripted suite never touches. Session-based per the Bachs' SBTM: charters (Explore X with Y to discover Z), 60-90 minute sessions, session sheets with TBS metrics, and the PROOF debrief. Bundles the exploration heuristics as references - Whittaker's seven tours, Kelly's FCC CUTS VIDS, Bach's SFDPOT. Broader than exploratory-charter-author, which writes one charter document: this owns the whole cycle from budgeting the available hours to debriefing what was found. Use when deciding what to explore with the time available, and how to run and record those sessions.

Install with skills.sh (any agent)

npx skills add testland/qa --skill exploratory-testing
View source

exploratory-testing

Overview

Session-Based Test Management (SBTM) is the dominant framework for managing exploratory testing as a measurable, accountable activity. It was developed by Jonathan Bach + James Bach (1999-2000) and is documented at satisfice.com/sbtm (opens in new window).

The unit of work is a session - a time-boxed (60-90 min) chunk of focused exploratory testing against a mission stated as a charter. Sessions produce session sheets that capture what happened in a structured-enough format for management to aggregate, while leaving room for the tester to learn freely.

Exploratory testing is not "clicking around". ISTQB defines it as testing where "tests are simultaneously designed, executed, and evaluated while the tester learns about the test object" (ISTQB CTFL Syllabus v4.0.1 §4.4.2, p.44 (opens in new window)). Human-driven testing has two modes, and this skill owns one of them:

Scripted manual testingExploratory testing
DesignedBefore execution, by someone elseDuring execution, by the person running it
Fixed in advanceSteps, data, expected resultA mission and a time box only
OutputPass/fail per case, signed and datedSession notes, bugs, issues, new charters
FindsDeviations from what was anticipatedWhat nobody thought to anticipate

For the scripted side (step-tables, Gherkin, UAT scripts, checklists), use manual-test-script-author. This skill covers the exploratory side end to end: charter, session, heuristics, debrief, review.

When to use

  • Onboarding a tester to SBTM vocabulary.
  • Authoring a charter - what fields belong, what mission framing to use (references/charter-template.md).
  • Picking which tour, heuristic, or oracle to apply in a session (routing table below).
  • Reviewing a session sheet for completeness (references/session-review-checklist.md).
  • Building a session-tracking dashboard.

How to use

  1. Write the charter - "Explore <area> with <tools> to discover <information>". Keep it a mission, not a test-case list. Ready-to-fill card: references/charter-template.md.
  2. Time-box the session - 60-90 min, one charter, uninterrupted; ~2-hour hard cap before focus degrades.
  3. Test and record - explore per the charter (applying tours + heuristics from the routing table below) and log Areas, Bugs, and Issues in the session sheet as you go. Full template: references/session-sheet-and-metrics.md.
  4. Track TBS - note the Test / Bug / Setup split (plus Opportunity / Idle). Verify: T should land in the healthy 60-80% range before you trust the session; if T < 50%, stop and fix the cause (stabilise the environment or narrow the charter), then re-run the session before aggregating.
  5. Close with PROOF - debrief Past, Results, Outlook, Obstacles, Feelings with the lead within 24h (full template + worked example: references/debrief.md; lead-side review rules: references/session-review-checklist.md).
  6. Aggregate on the dashboard - roll sessions up weekly (throughput, avg T%, charter progress) to drive strategy, not a bugs-per-session KPI.

The unit: a session

Per Jonathan and James Bach's SBTM paper (satisfice.com):

"A session is an uninterrupted block of reviewable, chartered test effort... A session may be short (around 60 minutes) or long (up to about 2 hours). Two hours is generally considered the upper limit because tester focus degrades."

Properties of a session:

  • Time-boxed: 60-90 minutes typical; absolute upper bound ~2 hours
  • Chartered: has a stated mission (what to explore, what to learn)
  • Uninterrupted: no context-switching to other work mid-session
  • Reviewable: produces a session sheet that the lead can review

The charter

A charter states the mission for one session. Per Bach:

Explore <area>
With <tools / resources>
To discover <information>

Example: "Explore the cart promo-stacking flow with manual sample inputs to discover discount-application bugs." More charter examples are in references/session-sheet-and-metrics.md; a full charter-card template (mission + areas + oracles + dimensions + tours + deliverables) is in references/charter-template.md.

Charters are not test cases. They state what to investigate and leave the how to the tester's judgment in the session.

A backlog of charters drives multi-session campaigns. The lead prioritises charters per the test strategy (test-strategy-author) and the risk register (risk-matrix).

Choosing a heuristic or tour

Each reference is a distinct lens; a strong session composes 2-3, never all of them. Route by what the session needs:

You needLensReference
A themed bug-hunting mission on a product you already knowWhittaker's seven tours (Feature, Money, Landmark, Intellectual, Bad-data, Configuration, Garbage collector's)references/tours.md
Recon on an unfamiliar product, before any charter existsKelly's FCC CUTS VIDS eleven touring questionsreferences/fcc-cuts-vids.md
Ideas for what to vary when stuck mid-sessionBach's SFDPOT (Structure, Function, Data, Platform, Operations, Time)references/sfdpot.md
To decide whether an observation is a bugBolton's HICCUPPS-F oracle catalogreferences/hiccupps-f.md
Which quality criteria the session evaluatesBach's CRUSSPIC STMPL thirteen criteriareferences/crusspic-stmpl.md
A ready-to-fill charter cardCharter template + quality rulesreferences/charter-template.md
To close and file the sessionPROOF debrief template + aggregationreferences/debrief.md
To review someone else's completed session sheetLead's review checklist + verdict rulesreferences/session-review-checklist.md
The session-sheet template, TBS metrics, dashboardSBTM artifacts in fullreferences/session-sheet-and-metrics.md

The catalogs complement each other: tours give the session a theme, SFDPOT gives variations, HICCUPPS-F interprets observations, and CRUSSPIC STMPL frames the criteria. Bach + Bolton's Rapid Software Testing curriculum teaches them as one toolkit.

The session sheet and metrics

Each session produces a session sheet whose sections are: Charter, Areas, session timings, TBS metrics, Data files, Test notes, Bugs, Issues (meta process problems), and the PROOF debrief. The TBS time-breakdown splits each session into T (test design + execution), B (bug investigation + reporting), and S (setup / overhead), plus Opportunity and Idle. Healthy ranges: T 60-80%, B 10-20%, S 10-15%; T under 50% signals trouble.

Full sheet template, the TBS definitions, and the cross-session dashboard metrics: references/session-sheet-and-metrics.md.

PROOF debrief

Every session closes with a PROOF debrief - the five-section report from the original Bach & Bach SBTM paper:

LetterStands for
PPast: what happened in this session
RResults: what was found
OOutlook: what's left to test; suggested follow-up charters
OObstacles: what blocked the tester
FFeelings: tester's qualitative read on quality

The debrief flow: the tester fills the PROOF template within 30 min of session end (memory fades fast); every Result links a bug ID or a confirmed-working item; Outlook names the recommended next charter (the chain to the next session); the lead reviews the debrief with the tester briefly (5-10 min) within 24h before the session sheet is filed, using the checklist in references/session-review-checklist.md. Feelings is the load-bearing field teams skip - the tester's qualitative judgment is signal no metric captures. Aggregated debriefs drive the next round of charters: which areas are well-covered, which are stale, which were never touched.

Full blank template, a fully worked debrief, the quarterly rollup table, and the attention tracker: references/debrief.md.

Worked example

A full 90-minute session (charter, timings, bugs, issues, TBS, PROOF outlook, and how the lead acts on it) is worked through in references/session-sheet-and-metrics.md.

Common confusions

ConfusionReality
"SBTM = ad-hoc testing."SBTM is structured: chartered, time-boxed, reviewed. Ad-hoc has none of those.
"Session sheet = test report."Session sheet captures the journey; test report aggregates outcomes.
"Charters = test cases."Charters state the mission; test cases prescribe steps.
"More TBS time = better tester."Wrong: T% is environment + charter health, not tester skill.
"Sessions need to find bugs to be valuable."Wrong: confirming coverage in a known-clean area is also valuable.
"Exploratory testing = unscripted."Distinct concepts: SBTM is management; unscripted execution is technique. SBTM sessions can be more or less scripted.

Anti-patterns

Anti-patternWhy it failsFix
Sessions > 2 hoursFocus collapses; quality dropsHard 2-hour cap
Multi-charter sessionsLoses focus; metrics ambiguousOne charter per session
Sessions tracked in a spreadsheet that no one reviewsEffort wasted; learning lostLead reviews each session within 24h
Charter as "test X"Doesn't direct explorationUse Explore X with Y to discover Z
TBS metrics never usedThrowaway dataAggregate weekly; drive process change
Bug-finding rate as KPIIncentivises shallow bug-huntingCharter throughput + dashboard health, not bugs-per-session
Session sheets without PROOFLose tester's qualitative signalAlways close with PROOF debrief
All heuristics in one sessionEleven shallow passes crowd out one useful oneCompose 2-3 lenses from the routing table

Limitations

  • Requires tester skill. The framework's value rises with tester expertise; it doesn't make bad testers good.
  • Cadence overhead. Sessions + sheets + reviews add coordination cost - worth it for medium-+ teams, overhead for solo testers.
  • Charter authoring is hard. Vague charters produce vague sessions; charter discipline pays off.
  • Dashboard interpretation requires context. Raw metrics (sessions/week, bugs/session) can mislead without charter framing.
  • Doesn't replace automated coverage. SBTM is for exploration, not regression - pair with automated suites.

References

Charter-card template

Deep reference for exploratory-testing SKILL.md. A ready-to-fill SBTM charter card: mission statement, scoped areas, applicable HICCUPPS-F oracles, SFDPOT dimensions to vary, and 1-3 recommended tours - so a junior or mid-level tester can start a session without further scaffolding.

The mission

The SBTM charter mission follows Bach's canonical three-part pattern (Bach J. + Bach J., "Session-Based Test Management", HP, 2000; landing page at satisfice.com/session-based-test-management):

Explore <area>
With <resources / tools / technique>
To discover <what you want to learn>

The "to discover" clause is load-bearing: per Bach's definition at satisfice.com/exploratory-testing, "exploratory testing means performing tests while learning things that may influence the testing." The mission must name what the tester is trying to learn - not just what to click.

Bad: "Explore the promo code feature." Good: "Explore the promo code apply flow with boundary inputs and stacking combinations to discover discount-calculation defects and error-state gaps."

Selecting the lenses

  1. HICCUPPS-F oracles (hiccupps-f.md (opens in new window)): pick 2-4 oracles that match the risk areas, and label them in the charter so the tester knows what standard of comparison to consult on suspect behaviour. Example for a "discount math" risk area:
    • C (Claims): Does the discount match the spec / acceptance criteria?
    • H (History): Did it work in the last release?
    • P (Product - internal consistency): Does cart total match receipt total?
    • S (Standards): Any locale-specific receipt-total statutes?
  2. SFDPOT dimensions (sfdpot.md (opens in new window)): pick the dimensions most relevant to the risk areas; annotate each with 1-2 concrete variation ideas. Example for an "expiry timing" risk area:
    • T (Time): apply promo 1 minute before expiry; apply 30 seconds after expiry.
    • D (Data): promo with no expiry set; promo with expiry = epoch zero.
  3. Tours (tours.md (opens in new window)): select 1-3 by mission type; note which SFDPOT dimension each tour primarily exercises.
Mission typeRecommended tours
New featureFeature tour + Bad-data tour
Post-change regressionLandmark tour + Garbage collector's tour
Complex business logicIntellectual tour
Money / pricing / billingMoney tour
Config / flags / rolesConfiguration tour

Cap at 3 tours per 90-minute session.

The charter card

Write to charters/<YYYY-MM-DD>-<kebab-feature>.md:

# Charter - <YYYY-MM-DD> - <kebab-feature>

**Mission:** Explore <area> with <resources> to discover <what to learn>.

**Created from:** <story / ticket / PR reference>
**Target build / SHA:** (tester fills before session)
**Time-box:** 90 min  (split into two 90-min charters if scope exceeds 7 areas)
**Tester:** ___________  **Date:** ___________

## Areas (3-7)

(derived from the risk areas supplied)

## Applicable HICCUPPS-F oracles

(2-4 oracles with one-line rationale each - see hiccupps-f.md)

## SFDPOT dimensions to vary

(3-5 dimensions with 1-2 concrete variation ideas each - see sfdpot.md)

## Recommended tours

(1-3 tours with brief rationale - see tours.md)

## Suggested test ideas

(5-10 concrete seed ideas derived from the above; the tester adapts these
moment-to-moment, not as a script)

## Out of scope

(explicit list: performance, security, cross-browser, a11y unless risk areas
named them; anything out of the feature's blast radius)

## Deliverables

- PROOF debrief at session end (Past, Results, Outlook, Obstacles,
  Feelings - see debrief.md).
- Bugs filed with oracle citation ("why is this a bug? - violates Claims oracle:
  spec says X, system does Y").
- Coverage note: which areas had time, which areas were blocked.

## Session log

(tester fills during the session)

## Sign-off

**Tester:** ___________  **End time:** ___________
**Time in test design:** ___ min  **In setup:** ___ min  **In bug investigation:** ___ min

Charter quality rules

A charter is not well-formed if any of these hold - fix before the session:

  • No feature description. "Test feature X" names a target without context; the charter needs the story / PR / one-paragraph summary it was created from.
  • No risk areas. The charter's areas derive from risk areas; without them the output is a vague scope, not a charter.
  • Time-box over 120 min. Split into two charters instead (per Bach's ~2-hour focus-degradation finding).
  • Multiple missions. "...and also cover the auth flow" is a second charter, not a second bullet. One mission per charter.

References

CRUSSPIC STMPL - Bach's quality-criteria mnemonic

View source (opens in new window)

CRUSSPIC STMPL - Bach's quality-criteria mnemonic

Deep reference for exploratory-testing SKILL.md. CRUSSPIC STMPL is James Bach's quality-criteria mnemonic. The thirteen criteria are enumerated under "Quality Criteria Categories" in Bach's Heuristic Test Strategy Model (HTSM v6.3, 2024-11-05), which defines a quality criterion as "some requirement that defines what the product should be" (HTSM (opens in new window)). The HTSM prints the criteria as named categories; the CRUSSPIC STMPL letters are the memory aid taught over them, not an acronym the HTSM itself spells out.

It overlaps with ISO/IEC 25010:2023 (the formal software- quality model) but offers a more practitioner-friendly mnemonic form that fits in an exploratory tester's working memory.

This reference is used when shaping a session's evaluation lens, and by the product-risk-register workflow (in qa-process's risk-matrix references/) when categorising risks by quality characteristic.

When to use

  • Authoring a charter - pick which quality criteria the session will evaluate.
  • Categorising risks in the product register (the product-risk-register reference of qa-process's risk-matrix).
  • Evaluating a vendor / framework (framework-choice-advisor in qa-process, including its vendor-evaluation reference).
  • Reviewing a release candidate - walk CRUSSPIC STMPL to confirm each criterion is addressed.

The thirteen criteria

CRUSSPIC - primary criteria

C - Capability

Does the system do what's claimed?

Functional correctness - the product performs its stated functions. Maps to ISO/IEC 25010 "Functional suitability."

Test heuristics: HICCUPPS-F's Claims oracle (hiccupps-f.md (opens in new window))

R - Reliability

Does the system work consistently over time?

  • Mean Time Between Failures (MTBF)
  • Fault tolerance (degraded mode, retry, failover)
  • Recovery time + data preservation
  • Idempotency under retry

Maps to ISO 25010 "Reliability."

U - Usability

Can the user figure out how to use it?

  • Learnability for new users
  • Efficiency for repeat users
  • Memorability after time away
  • Error-recovery from mistakes
  • Satisfaction (qualitative)

Per Nielsen's usability heuristics.

S - Security

Is the system safe from misuse?

  • Confidentiality (PII / secrets not exposed)
  • Integrity (data not modified by attackers)
  • Availability (DoS resistance)
  • Authentication + authorization
  • Auditability

Maps to ISO 25010 "Security." Composes with qa-security-scanning, qa-test-data-privacy.

S - Scalability

Does the system grow with load?

  • Vertical scaling (bigger machines)
  • Horizontal scaling (more machines)
  • Data-volume scaling (more rows, bigger blobs)
  • User-count scaling (more concurrent users)

Composes with qa-load-testing.

P - Performance

Is the system fast enough?

  • Response latency (p50, p95, p99)
  • Throughput
  • Resource efficiency (CPU, memory, network, disk)
  • Startup / cold-start time

Distinct from Scalability - performance is "fast for the user"; scalability is "still fast when load grows." Composes with qa-load-testing.

I - Installability

Can the user install + configure the system?

  • Installer / setup wizard quality
  • Configuration documentation
  • Default values appropriate
  • Upgrade path from prior versions
  • Uninstall completeness

Maps to ISO 25010 "Portability - Installability."

C - Compatibility

Does the system work alongside other things?

  • OS / browser compatibility matrix
  • Backwards-compatibility (old data, old clients)
  • Forwards-compatibility (newer data, newer clients)
  • Coexistence (same machine, same network, with other software)
  • Interoperability (API consumers)

Maps to ISO 25010 "Compatibility."

STMPL - secondary (operational) criteria

S - Supportability

Can the support team diagnose + fix issues?

  • Log quality (useful, structured, searchable)
  • Observability (traces, metrics)
  • Self-service support materials (knowledge base)
  • Escalation paths
  • Reproducibility from logs alone

T - Testability

Can the testers verify the system?

  • Test hooks / instrumentation
  • Deterministic-mode toggles
  • State-inspection endpoints
  • Test-data-management hooks
  • Mock-able external dependencies

M - Maintainability

Can the team change the system?

  • Code clarity / structure
  • Documentation
  • Test coverage
  • Dependency management
  • Tech debt accumulation

Maps to ISO 25010 "Maintainability."

P - Portability

Can the system move between environments?

  • Cloud-provider portability (AWS / GCP / Azure)
  • OS portability
  • Container / VM portability
  • Data portability (export + import)
  • Configuration portability

Maps to ISO 25010 "Portability."

L - Localizability

Can the system be adapted to other languages + cultures?

  • Translation support
  • Right-to-left layout
  • Locale-specific formatting (dates, numbers, currency)
  • Cultural appropriateness (icons, colours, idioms)
  • ICU MessageFormat / plural rules

Composes with qa-localization.

Mapping to ISO/IEC 25010

ISO 25010 has 8 top-level characteristics:

ISO 25010CRUSSPIC STMPL
Functional suitabilityC (Capability)
ReliabilityR (Reliability)
UsabilityU (Usability)
SecurityS1 (Security)
Performance efficiencyP1 (Performance), S2 (Scalability)
CompatibilityC2 (Compatibility)
MaintainabilityM (Maintainability), T (Testability), S3 (Supportability)
PortabilityI (Installability), P2 (Portability), L (Localizability)

CRUSSPIC STMPL is more granular; ISO 25010 is more formal. The two are complementary, not substitutable. Per ISO/IEC 25010:2023 (cite by stable ID).

Worked example - applying CRUSSPIC STMPL to a release review

Pre-release review:

## Release v3.0 quality-criteria walkthrough

- **C - Capability:** Functional acceptance tests 98% pass. ✓
- **R - Reliability:** Last 30 days uptime 99.94% (SLA: 99.9%). ✓
- **U - Usability:** User research session N=8; 7/8 completed
  signup without help (target: 6/8). ✓
- **S - Security:** Last pen-test 2026-03; all critical findings
  fixed; SAST + DAST clean. ✓
- **S - Scalability:** k6 load test at 10x expected peak passed
  with p99 < 800ms. ✓
- **P - Performance:** p95 latency 220ms (target 300ms). ✓
- **I - Installability:** New install flow 5 steps; user testing
  4/5 completed. ✓
- **C - Compatibility:** Browser matrix Chrome / Firefox / Safari
  all current + N-1 tested. ✓
- **S - Supportability:** All log lines structured; trace IDs
  end-to-end. ✓
- **T - Testability:** E2E suite runs in 18 min; deterministic
  mode available. ✓
- **M - Maintainability:** Sonar tech-debt rating A. ✓
- **P - Portability:** Cloud-portable (Terraform); no AWS-specific
  primitives. ✓
- **L - Localizability:** All UI strings extracted to ICU
  messages; 7 locales supported. ✓

Verdict: cleared for release.

Anti-patterns

Anti-patternWhy it failsFix
Treating CRUSSPIC STMPL as flatSome criteria matter much more for some products (Security for fintech, Localizability for global B2C)Weight per product context
Confusing Performance with Scalability"Fast" doesn't mean "stays fast under load"Test both independently
Treating Maintainability + Testability as "developer concerns"They're quality criteria the tester evaluatesInclude in release walkthrough
Skipping SupportabilityReleased system unmaintainable in productionAlways walk S
One person evaluates all 13Inter-criterion expertise differsDistribute walkthrough across team
Walkthrough at release time onlyQuality criteria degrade silently between releasesContinuous monitoring per criterion

Limitations

  • Subjective weighting. Criteria importance differs per product / domain / regulatory context.
  • Some criteria are evaluatable only in production. Reliability
    • Supportability need real usage data.
  • Mnemonic overload. Thirteen letters is at the edge of working memory; testers often print the catalog.
  • Doesn't replace ISO 25010 formal model. For regulated / compliance contexts use ISO 25010 + cite formally; use CRUSSPIC STMPL as the working-memory mnemonic.

References

PROOF session debrief - template, worked example, aggregation

View source (opens in new window)

PROOF session debrief - template, worked example, aggregation

Deep reference for exploratory-testing SKILL.md. The Session-Based Test Management (SBTM) framework ends every session with a PROOF report - a five-section debrief the team can aggregate, compare, and act on. Skip it and the session's findings live only in the tester's head: no team learning, no audit trail, no follow-up. This reference holds the template, a fully worked example, and the aggregation conventions.

When to use

  • After every charter session.
  • After a bug bash cohort completes its 90-min slot per bug-bash-facilitator.
  • After a free-form tester exploration (even without a formal charter, the PROOF structure converts the session to an artifact).

PROOF format

The acronym from the original Bach & Bach SBTM paper:

LetterSectionWhat it captures
PPastWhat was tested - areas covered, paths walked, tours applied.
RResultsWhat was learned - confirmed-working items, surprises, novel observations.
OOutlookWhat's left - areas NOT covered; what to explore next session.
OObstaclesWhat slowed the session - broken setup, missing test data, environment instability.
FFeelingsTester's qualitative read on product quality (confident / uneasy / unsure).

Feelings is the load-bearing field teams skip. Keep it - the tester's qualitative judgment is signal no metric captures.

The debrief template

Fill one per session, within 30 min of session end. The blank spine:

# Session debrief - `<session-id>`

**Charter:** [link]   **Tester:** ____   **Date:** ____
**Time-box:** 90 min   **Actual:** ___ min   **Build / Env:** ____

## Past - what was tested
Tours applied (per tours.md); areas covered;
notable paths walked.

## Results - what was learned
Confirmed-working items; bugs found (with bug IDs); surprises;
confirmed-fixed vs prior session.

## Outlook - what's left
Areas not covered (out of time / scope); recommended next charter;
open questions for PM / dev.

## Obstacles
Setup pain; environment instability; recommendations for next session.

## Feelings
Quality of attention; confidence in the feature; unease about untested
areas; recommendation to the release manager.

## Time accounting (3-bucket)
| Bucket | Minutes |
|---|--:|
| Test design | __ |
| Setup | __ |
| Bug investigation | __ |
| **Total actual** | __ |

A high setup % is a signal the environment or test data needs investment, not that the tester is slow.

A fully worked debrief (promo-code checkout session)

# Session debrief - `<session-id>`

**Charter:** [link to charter]
**Tester:** _______________
**Date:** _______________
**Time-box:** 90 min   **Actual:** ___ min
**Build / Environment:** _______________

## Past - what was tested

**Tours applied:**
- [x] Money tour (per tours.md)
- [x] Bad-data tour
- [ ] Configuration tour (skipped - out of time)

**Areas covered:**
- Promo code input field - full coverage including SQL/XSS payloads
- Promo discount math - 10% / 50% / 100% / fractional cent edge cases
- Promo + tax interaction - covered for US tax states only

**Paths walked (notable ones):**
- Apply WELCOME10 to $24.99 cart -> $22.49 (pass)
- Apply two stackable promos -> second silently overrides first (BUG-987)
- Apply expired promo -> graceful error message (pass)

## Results - what was learned

**Confirmed working:**
- Single-promo apply
- Promo code expiration enforcement
- Free-shipping promo

**Bugs found:** (with bug IDs)
- BUG-987: Stacking two promos doesn't combine - second silently overrides first.
- BUG-988: SQL injection in promo input field returns 500 instead of 400.
- BUG-989: $0.01 cart with 50% off rounds to $0.00 instead of $0.01.

**Surprises:**
- Discount is applied to subtotal BEFORE tax, but the original SOW
  said after-tax. Need to clarify with PM.
- "WELCOME10" code is case-sensitive; "welcome10" silently rejected
  with no helpful message.

**Confirmed-fixed (vs prior session):**
- Previous BUG-832 (promo input losing focus) is fixed. (pass)

## Outlook - what's left

**Areas not covered (out of time / scope):**
- EU tax cases (covered separately by next session's charter).
- Multi-currency promo behavior.
- Promo + subscription billing.

**Recommended next charter:**
- "Explore the EU tax + promo interaction" (90 min).
- "Explore promo + subscription billing edge cases" (60 min).

**Open questions for PM / dev team:**
- Confirm: discount before vs after tax (cited as "before" in
  current implementation; SOW says "after").
- Confirm: should case-insensitive promo codes be supported?

## Obstacles

**Setup pain:**
- Stripe test card kept timing out at checkout - added 5 min to
  the session.
- Required test promo codes weren't pre-seeded; had to create them
  manually.

**Environment instability:**
- Staging was down for ~10 min mid-session; lost momentum.

**Recommendations for next session:**
- Pre-seed promo codes via a fixture per `synthetic-data-toolkit`.
- Verify staging is up before session start.

## Feelings

**Quality of attention this session:** Strong (focused throughout;
caught the BUG-987 cluster early which sustained interest).

**Confidence in the feature:** Mixed. The single-promo path is
solid; the multi-promo path has architectural issues that aren't
just bugs (SOW ambiguity on discount-before-tax suggests the
business hasn't fully decided).

**Unease about untested areas:** Moderate. Multi-currency promos
weren't touched; gut says there are bugs there.

**Recommendation to release manager:** Block release until
BUG-987 (stacking) and BUG-988 (SQL injection) are fixed.
BUG-989 (rounding) is low impact; can ship with known-issue note.

## Time accounting (3-bucket)

| Bucket            | Minutes |
|-------------------|--------:|
| Test design       |      35 |
| Setup             |      25 |
| Bug investigation |      30 |
| **Total actual**  |      90 |

Aggregation across sessions

Individual debriefs are useful; aggregating them surfaces patterns - which areas are well-covered, which are stale, which have never been touched, and where bugs cluster. Charter authoring uses the rollup to pick where the next session should focus.

## Quarterly debrief rollup - Q2 2026

**Sessions completed:** 47
**Bugs raised:** 138
**Average session: 90-min charter, 3-bucket: 38 / 28 / 24**

### Areas by coverage

| Area                       | Sessions | Bugs found | Last covered |
|----------------------------|---------:|-----------:|--------------|
| Checkout - promo flow       |    8    |    34     | 2026-04-28  |
| Checkout - payment          |    6    |    19     | 2026-05-02  |
| Account - subscription       |    4    |    12     | 2026-05-05  |
| Account - profile            |    3    |     8     | 2026-04-15  |
| Admin panel                  |    1    |     2     | 2026-03-20  |  (stale)
| Reports                      |    0    |     0     | (never)      |  (uncovered)

### Action items from rollup

- Schedule sessions for Admin panel + Reports areas (under-covered).
- Subscription area surfaced 12 bugs - investigate root-cause
  pattern.
- Average setup time (28 min) is high - invest in fixture tooling.

The rollup shows what's been explored vs what's stale vs what's never been touched. Charter authoring uses this to pick where the next session focuses.

Quality-of-attention signal

The Feelings section produces a per-session subjective signal. Track it over time:

SessionFeelings (numeric: 1-5)Notes
ses-2014Strong; BUG cluster found early.
ses-2022Weak; tester sick / distracted.
ses-2035Excellent; new tester, fresh eyes.

When attention is consistently low across a tester / area, change something - different tester, different time-box, different tour, different scope.

Anti-patterns

Anti-patternWhy it failsFix
Skipping the Feelings sectionLoses the qualitative signal; rollup is metric-only.Always fill it.
Debrief written days laterMemory faded; details lost.Author within 30 min of session end.
Debrief without bug IDsFindings can't be tracked; team can't follow up.Every "Result" links to a bug ID OR is logged as a quirk.
One-line Past section ("tested promo")Coverage gap invisible at rollup time.List tours, areas, paths walked.
No Outlook sectionNext session has no continuity; same areas re-explored."Recommended next charter" is the chain to next session.
Aggregating without actingRollup data sits; team doesn't reallocate sessions.Action items per rollup.
Debrief in chat / Slack DMsNot searchable; not in version control.Markdown file in docs/sessions/<session-id>.md (or wiki / Notion DB).

Limitations

  • Self-reported. Honesty is load-bearing. A tester who consistently overstates attention quality skews the rollup.
  • Time accounting is approximate. The 3-bucket split rounds to the nearest 5 min; that's fine.
  • Aggregation is manual. A rollup script can compute coverage % but the action items need a human's call.
  • Per-team conventions vary. Some teams add a 6th letter (PROOFS - Stakeholders) for who needs to see the report. Adapt the template per team norms.

References

  • Bach, J. & Bach, J., Session-Based Test Management (HP, 2000; PDF at satisfice.com/download/session-based-test-management) - PROOF debrief format origin, three-bucket time accounting.
  • tours.md (opens in new window) - the tours catalogued in the Past section.
  • session-review-checklist.md (opens in new window) - the lead's review pass over a filled debrief.
  • bug-bash-facilitator - multi-cohort bug bash inherits this debrief format.

FCC CUTS VIDS - Kelly's touring heuristic

View source (opens in new window)

FCC CUTS VIDS - Kelly's touring heuristic

Deep reference for exploratory-testing SKILL.md. FCC CUTS VIDS is Michael Kelly's touring heuristic, published on his blog on 20 September 2005. It names eleven tours, each a short reconnaissance pass over an application with one question in mind. Kelly introduced it as a companion to his test-reporting heuristic, writing "I think I will need something similar for application touring. Here is my attempt: FCC CUTS VIDS" (michaeldkelly.com (opens in new window)).

The tours answer what does this product even consist of - they are aimed at a tester who does not yet know the application. That is the axis that separates them from Whittaker's seven tours (tours.md (opens in new window)), which come from Exploratory Software Testing (2009) and frame a themed bug hunt on a product the tester already understands. Kelly's tours precede that work by four years and are used earlier in the lifecycle: recon first, mission after.

When to use

  • Onboarding onto an unfamiliar product, before any charter is written.
  • Inheriting an area nobody on the team has tested recently.
  • Opening a first session on a feature whose shape is unknown, to decide what is worth chartering at all.
  • Filling a gap mid-session when the tester realises they cannot answer a basic question about the system.

Do not reach for this once the product is well understood; at that point a themed mission from tours.md (opens in new window) is the better tool.

How to use

  1. Pick a target. One application, or one area of it, in a sentence.
  2. Run a subset of tours, not all eleven. Each is a short pass with one question. Three or four chosen for the unknowns that actually matter beat a mechanical sweep through the full list.
  3. Take notes per tour so the pass produces a record, not just a feeling of familiarity.
  4. Convert the gaps into charters. A tour that raises more questions than it answers has found the area worth a session. Charter and time-box those per the umbrella SKILL.md.

The eleven tours at a glance

Each description is Kelly's own wording.

GroupTourKelly's description
FCCF - Feature"Move through the application and get familiar with all the controls and features you come across."
C - Complexity"Find the five most complex things about the application."
C - Claims"Find all the information in the product that tells you what the product does."
CUTSC - Configuration"Attempt to find all the ways you can change settings in the product in a way that the application retains those settings."
U - User"Imagine five users for the product and the information they would want from the product or the major features they would be interested in."
T - Testability"Find all the features you can use as testability features and/or identify tools you have available that you can use to help in your testing."
S - Scenario"Imagine five realistic scenarios for how the users identified in the user tour would use this product."
VIDSV - Variability"Look for things you can change in the application - and then you try to change them."
I - Interoperability"What does this application interact with?"
D - Data"Identify the major data elements of the application."
S - Structure"Find everything you can about what comprises the physical product (code, interfaces, hardware, files, etc...)."

The User and Scenario tours are ordered: Scenario builds on the personas the User tour produced, so run User first.

The eleven tours in full

Each tour below quotes Kelly's own one-line description verbatim. The prompts and worked example under each are elaboration for this catalog, not Kelly's text.

Running example throughout: a self-serve expense-reporting web app the tester has never opened before.

FCC - the product shape

F - Feature tour

"Move through the application and get familiar with all the controls and features you come across."

Prompts

  • What is on every screen reachable without special permissions?
  • Which controls are visible but disabled, and under what condition do they enable?
  • What exists in menus that has no obvious entry point in the UI?

Worked example. Clicking every nav item surfaces a "Recurring expenses" screen that appears in no onboarding material. Nobody on the team mentions it. That absence is the finding.

C - Complexity tour

"Find the five most complex things about the application."

Prompts

  • Which screen has the most fields, states, or conditional logic?
  • Where does the product ask the user to make the hardest decision?
  • What would be hardest to explain to a new colleague?

Worked example. The five: multi-currency conversion, approval-chain routing, receipt OCR, the mileage calculator, and per-country tax rules. Complexity concentrates risk, so this list is usually the first draft of a charter backlog.

C - Claims tour

"Find all the information in the product that tells you what the product does."

Prompts

  • What do the marketing pages, in-product tooltips, help centre, and release notes promise?
  • Where do two of those sources disagree?
  • What does the product claim about limits, formats, or timing?

Worked example. The pricing page says receipts are processed "instantly"; the in-app tooltip says "within a few minutes". Both are claims, and they conflict. That conflict is testable, and it is also material for hiccupps-f.md (opens in new window), where Claims is one of the oracles.

CUTS - the usage shape

C - Configuration tour

"Attempt to find all the ways you can change settings in the product in a way that the application retains those settings."

Prompts

  • What is settable per user, per team, and per organisation?
  • Which settings survive logout, and which silently reset?
  • What is configurable only by an administrator or only by support?

Worked example. The default currency is settable per user, but resets to the org default after a session expires. Persistence is the part of the tour that matters: a setting that does not stick is a different product than the one the settings screen advertises.

U - User tour

"Imagine five users for the product and the information they would want from the product or the major features they would be interested in."

Prompts

  • Who are five distinct people who touch this, and what does each want from it?
  • Which of them never log in but are affected by it anyway?
  • Whose needs conflict?

Worked example. A field engineer filing from a phone, a manager approving in bulk, a finance analyst exporting for reconciliation, an auditor reading historical records, and an admin configuring policy. The analyst and the auditor want opposite things from data retention.

Run this tour before the Scenario tour, which depends on its output.

T - Testability tour

"Find all the features you can use as testability features and/or identify tools you have available that you can use to help in your testing."

Prompts

  • What logs, debug views, health endpoints, or admin panels exist?
  • Can state be set up directly, or only through the UI?
  • What is available for generating or resetting data?

Worked example. An admin "impersonate user" action removes the need to hold five sets of credentials, and a CSV import can seed a hundred expenses in one step. Both cut session setup time, which is the constraint the SBTM TBS metrics care about (session-sheet-and-metrics.md (opens in new window)).

S - Scenario tour

"Imagine five realistic scenarios for how the users identified in the user tour would use this product."

Prompts

  • What does each persona from the User tour do end to end, on a normal day?
  • Which scenario crosses the most features?
  • What does the unhappy version of each scenario look like?

Worked example. The field engineer photographs a receipt in a basement with no signal, files it later, and the manager approves it from an email link while travelling. That scenario crosses offline capture, sync, deep linking, and approval - four areas one tour surfaced together.

VIDS - the data and system shape

V - Variability tour

"Look for things you can change in the application - and then you try to change them."

Prompts

  • Which fields accept free input, and what do they accept?
  • What can be reordered, renamed, deleted, or bulk-edited?
  • What changes state as a side effect of changing something else?

Worked example. An expense category can be renamed after reports referencing it are submitted. Whether historical reports show the old or new name is a real question the tour raises but does not answer.

Note the overlap with SFDPOT (sfdpot.md (opens in new window)): variability is about finding what can change, SFDPOT is about systematically varying it once found.

I - Interoperability tour

"What does this application interact with?"

Prompts

  • What does it authenticate against, import from, export to, or notify?
  • Which integrations are optional, and what breaks when one is off?
  • What talks to it that the team does not own?

Worked example. Single sign-on, a corporate card feed, an accounting export, and an email notification service. The card feed is the one nobody on the team controls, which makes it the first candidate for contract-level attention.

D - Data tour

"Identify the major data elements of the application."

Prompts

  • What are the main entities, and how do they relate?
  • Which fields are required, unique, or bounded?
  • What data outlives the user who created it?

Worked example. Expense, Report, Approval, Policy, User. An Approval references a User who may later be deactivated, which raises the question of what an audit view shows for a departed employee.

S - Structure tour

"Find everything you can about what comprises the physical product (code, interfaces, hardware, files, etc...)."

Prompts

  • What are the deployable pieces, and where do they run?
  • What file formats, APIs, and storage does it own?
  • What is the client, and what is the server?

Worked example. A React front end, a REST API, a background OCR worker, object storage for receipt images, and a nightly export job. The worker and the export job have no UI, so nothing in the Feature tour would have found them.

Turning a tour into a charter

A tour produces questions; a charter answers one. When a tour surfaces something worth pursuing, charter it with a stated mission and a time box (charter-template.md (opens in new window)). The Complexity and Claims tours tend to generate the most charter-worthy material, because both surface places where the product's stated behaviour and its actual behaviour can diverge.

Anti-patterns

Anti-patternWhy it failsDo instead
Running all eleven tours on every productThe heuristic is a menu, not a checklist; eleven shallow passes crowd out one useful onePick the tours that target what is actually unknown
Treating a tour as a test passA tour builds familiarity; it is not coverage and finds bugs only incidentallyCharter a session for the risks the tour surfaced
Confusing these with Whittaker's toursDifferent author, different set, different lifecycle stageKelly's eleven for recon, tours.md (opens in new window) for themed missions
Attributing the mnemonic to James BachIt is Kelly's, from his own blogCite michaeldkelly.com
Touring without notesThe familiarity evaporates and the next tester starts overRecord findings per tour so gaps become charters

Limitations

  • The tours are a learning aid. They do not establish coverage, and a product that has been toured is not a product that has been tested.
  • Kelly's original post gives one sentence per tour and no worked example; the prompts in this catalog are elaboration, not Kelly's text, and are marked as such.
  • The set is from 2005 and predates mobile, cloud, and API-first products. The Interoperability and Structure tours carry most of the weight for those, but the list has no tour aimed squarely at, say, a third-party identity provider.

References

HICCUPPS-F - Bolton's oracle heuristic

View source (opens in new window)

HICCUPPS-F - Bolton's oracle heuristic

Deep reference for exploratory-testing SKILL.md. HICCUPPS-F is Michael Bolton's oracle heuristic - a mnemonic for the kinds of references a tester consults to decide whether an observation is a problem. It's published at developsense.com/blog/2012/07/few-hiccupps (opens in new window).

The point: a "bug" is a relationship between an observation and some expectation. Different expectations come from different oracles. HICCUPPS-F gives the tester a checklist of oracle types to consult before deciding "no oracle ⇒ probably not a bug" or "oracle says X ⇒ behaviour Y is wrong."

When to use

  • Mid-session: tester sees behaviour X, asks "is this a bug?" - walk HICCUPPS-F to find the matching oracle.
  • Authoring a charter: pre-identify which oracles the session should consult (charter-template.md (opens in new window)).
  • Bug-report review: the bug report's "why is this a bug?" must cite at least one HICCUPPS-F oracle.
  • Onboarding: this is the canonical "where do test expectations come from?" vocabulary.

The nine oracles

Per Bolton's published catalog:

H - History

Does the system's current behaviour match what it did before?

Behaviour regressed from a known prior version = bug. Sources:

  • Git blame on the relevant code path
  • Previous release notes / changelog
  • Old screenshots / videos in user-acceptance archives
  • The release-1.2 regression suite (qa-test-impact-analysis)

I - Image

Does the behaviour match the company's brand and reputation?

The product's overall feel: error messages should not be hostile, loading states should look professional, copy should be on-brand. Sources:

  • Brand guidelines / style guide
  • Marketing materials
  • Comparable user touchpoints (the company's other products)

C - Comparable products

Does the behaviour match what competitors / peers do?

Industry conventions. A login form that lacks "forgot password" when every competitor has one. Sources:

  • Competitor screenshots
  • Industry benchmark reports
  • Stack Overflow / dev forum norms

C - Claims

Does the behaviour match what stakeholders said it would do?

The spec, the requirements document, the customer-promise email, the sales-deck slide. Sources:

  • Requirements / acceptance criteria
  • Spec docs
  • Sales / marketing collateral
  • Customer escalation transcripts

U - Users' desires

Does the behaviour match what users actually want / need?

Users may want something different than what the spec says. Sources:

  • User-research interviews
  • Support-ticket aggregations
  • NPS / CSAT comments
  • Direct customer feedback

P - Product (internal consistency)

Does the behaviour match other behaviours in the same product?

The settings page uses a save button; the profile page does auto-save. The same data field is formatted differently across two screens. Sources:

  • The product itself - explore adjacent areas
  • Internal style guide
  • Component library / design system

P - Purpose

Does the behaviour match the actual reason the feature exists?

The feature exists to help X do Y; the behaviour doesn't help X do Y. Sources:

  • Original feature design doc
  • OKR / business justification
  • Customer success outcomes

S - Standards / statutes

Does the behaviour comply with relevant standards + regulations?

WCAG accessibility, GDPR data handling, ISO 25010 quality characteristics, PCI-DSS for payments, HIPAA for health, RFC for network protocols. Sources:

  • The applicable standard (cite the section)
  • Regulatory body guidance
  • Industry compliance reports

F - Familiar problems

Have we seen this kind of bug before, in this or other systems?

Pattern-matching against known bug classes:

  • Off-by-one errors
  • Time-zone edge cases (DST, leap day, leap second)
  • Unicode normalisation (NFC vs NFD)
  • Cache invalidation
  • Race conditions on shared state

Sources:

  • The team's bug history (qa-defect-management)
  • OWASP Top 10 / CWE Top 25
  • Industry bug catalogs (Beizer, Kaner, Myers)

Worked example - applying HICCUPPS-F mid-session

Observation: Cart total shows $24.99 when promo "TAX10" applied,
but receipt PDF shows $25.49.

Walk HICCUPPS-F:

- H (History): Old screenshots from v1.4 show consistent totals.
  → This is a regression. **BUG.**
- I (Image): Inconsistent values reflect badly on the brand. Even
  if there's no other oracle, this fails Image.
- C (Comparable): Stripe / PayPal flows always show consistent
  totals across cart + receipt. **Industry convention violated.**
- C (Claims): Promo code spec says "TAX10 applies 10% before tax."
  Cart applies it before tax (correct); receipt applies it after
  tax (incorrect). **Spec violated.**
- U (Users' desires): Users will dispute the $0.50 difference
  with support. **Visible to customer.**
- P (Product): Cart and receipt should be consistent at minimum.
  **Internal consistency broken.**
- Purpose: Promo feature exists to encourage purchase; mismatched
  totals erode trust. **Purpose undermined.**
- S (Standards): Statutory? Possibly - depends on locale (some
  jurisdictions require receipts to match displayed totals).
- F (Familiar problems): Rounding-order bug; classic off-by-cent
  pattern. **Known bug class.**

Conclusion: Multiple oracles agree this is a bug. File
high-priority.

Anti-patterns

Anti-patternWhy it failsFix
Consulting only one oracleMisses bugs visible from other anglesWalk all 9; even briefly
"I don't see an oracle ⇒ probably fine"Some bugs only one oracle catches (F - familiar problem patterns from elsewhere)Look at F (familiar problems) before concluding "no oracle"
Charter doesn't pre-state expected oraclesSession aimlessCharter should hint at applicable oracles ("explore X with HICCUPPS-F focusing on Claims + Users + Standards")
Bug report without HICCUPPS-F citationReport's "why is this a bug?" weakEvery bug report should cite at least one oracle from HICCUPPS-F
Treating Standards as final authorityStandards lag; users / purpose may indicate the real bugUse all 9 as inputs, not as final-word hierarchy
Skipping F (familiar problems) for new productMost "new" bugs are familiar patterns from other systemsAlways check F

Limitations

  • Vocabulary, not algorithm. HICCUPPS-F doesn't tell the tester what to test; it tells them what kinds of references to consult.
  • Oracles can disagree. History says X but Claims says Y; the tester must reconcile.
  • Skill-dependent application. A senior tester walks all 9 fluently; a junior may need to refer to a checklist.
  • Some oracles cost-prohibitive. User research (U) takes research; not feasible for every bug-or-not decision mid-session.

References

Session-sheet review checklist

View source (opens in new window)

Session-sheet review checklist

Deep reference for exploratory-testing SKILL.md. The lead's (or a peer's) review pass over a completed SBTM session sheet, run before the sheet is filed - the reviewer half of the debrief loop that debrief.md (opens in new window) starts.

The review pass

  1. Locate the sheet. Review the filed debrief file or the inline text. Refuse to review if no session sheet is present.

  2. Check each PROOF field. Per Jonathan and James Bach's SBTM framework (satisfice.com/download/session-based-test-management (opens in new window)), rate each as Complete, Thin, or Missing. Minimum bar: Past names areas + tours; Results has bug IDs or confirmed-working items; Obstacles names blockers or states "none"; Outlook lists uncovered areas and one next charter; Feelings states product-confidence level (confident / mixed / uneasy) - not a number.

  3. Flag thin Feelings specifically. Feelings is the load-bearing field (debrief.md (opens in new window)). A single word or numeric rating is Thin. Coach: rewrite to state product-confidence level (confident / mixed / uneasy) and attention quality.

  4. Evaluate the TBS split. Healthy thresholds (session-sheet-and-metrics.md (opens in new window), same SBTM source above): T 60-80%, B 10-20%, S 10-15%.

    • S > 30%: environment problem; recommend pre-seeding test data and verifying the environment before the next session.
    • T < 50%: charter too broad, or environment collapsed; split the charter or fix the environment before the next session.
    • If bucket percentages are absent: flag as a missing metric.
  5. Recommend the next charter. From the Outlook's uncovered areas, synthesize one charter in Explore X with Y to discover Z form (charter-template.md (opens in new window)). If Outlook is empty, derive from Results surprises or Obstacles blockers.

Review output format

Sections in order:

  1. PROOF completeness - one table row per field (Field / Status / Coaching note).
  2. TBS health - one line per bucket with [OK / WARNING] label.
  3. Recommended next charter - Explore X with Y to discover Z.
  4. Verdict - READY TO FILE or REVISE FIRST with one-sentence priority fix.

Hard rules

  • No Charter field present: not a valid SBTM session sheet - do not review.
  • Feelings section absent (not just thin): the sheet cannot be marked READY TO FILE per the PROOF contract.
  • Results section empty: no next-charter recommendation can be derived.

References

SBTM session sheet + metrics in full

View source (opens in new window)

SBTM session sheet + metrics in full

Deep reference for exploratory-testing SKILL.md. Consult when authoring or reviewing a session sheet, or building the cross-session dashboard. Holds the canonical sheet template, the TBS time-breakdown, and the dashboard metrics.

The session sheet

Each session produces a session sheet. Bach's canonical structure (satisfice.com):

# Session sheet - YYYY-MM-DD - <tester>

## Charter

Explore <area> with <tools> to discover <information>.

## Areas

- (system area 1)
- (system area 2)
- ...

## Session start / duration / setup time / focus

- Started: 14:00
- Duration: 90 min
- Setup time: 10 min
- Charter time: 70 min
- Bug-investigation time: 10 min
- Opportunity time: 0 min

## TBS metrics (time-breakdown)

- Test design + execution: 70%
- Bug investigation + reporting: 11%
- Setup / overhead: 11%
- Opportunity: 0%
- Idle / interruption: 8%

## Data files

- screenshots/2026-05-20-14-15.png
- har/2026-05-20-14-22.har

## Test notes

(narrative of what was tested, in tester's own words; includes
tours applied, heuristics applied, hypotheses formed)

## Bugs (file later)

- B-001: Promo "STACK50" applies after tax instead of before;
  reproduces 3/3. Captured at 14:35.
- B-002: Empty cart + apply promo → page error, not graceful message.

## Issues (meta - testing-process problems)

- Cannot get to step 4 in flow without a paid customer account;
  test data unavailable. Blocking 40% of charter scope.

## PROOF debrief

(See debrief.md)

TBS metrics - time breakdown

Per Bach's SBTM paper, sessions decompose into:

CategoryDefinition
T (Test)Time spent on test design + execution per the charter
B (Bug)Time spent investigating + reporting bugs
S (Setup)Time setting up the environment / test data / tools

Plus often-included:

  • Opportunity: unrelated bugs found by chance; investigated outside charter scope
  • Idle: waiting on a build / response

Healthy session: T 60-80%, B 10-20%, S 10-15%. Skewed sessions (T < 50%) signal problems - environment instability, charter too broad, etc.

Dashboard metrics - across sessions

Per Bach's SBTM Reporting paper (satisfice.com), the lead views:

MetricWhat it tells
Sessions per weekThroughput
Avg T% across sessionsEnvironment / charter-scope health
Bugs per sessionFind rate (interpret carefully - not all sessions should find bugs)
Charters complete / in-progress / blockedCoverage progress
Charter-to-bug ratioQuality of charter framing (too broad = many small bugs; too narrow = few)

These feed the testing-strategy review at sprint planning.

Charter examples

A charter states the mission for one session (Explore <area> with <tools> to discover <information>). Sample charters:

Charter
"Explore the cart promo-stacking flow with manual sample inputs to discover discount-application bugs."
"Explore the password-reset endpoint with the OWASP Top 10 list to discover injection / SSRF vulnerabilities."
"Explore the checkout error states with a flaky-network proxy to discover retry behaviour issues."
"Explore the admin dashboard's role-permission UI with three test users to discover authorization-leak bugs."

Worked example - one 90-minute session

**Charter:** Explore the cart promo-stacking flow with manual sample
inputs to discover discount-application bugs.

**Timings:** Started 14:00; 90 min; 10 min setup; 70 min charter;
10 min bug investigation.

**Bugs:** B-001 - "STACK50" applies after tax instead of before,
reproduces 3/3.

**Issues:** Cannot reach step 4 without a paid account; blocks ~40%
of charter scope.

**TBS:** T 70% / B 11% / S 11% / Idle 8% - healthy.

**PROOF Outlook:** recommend a follow-up charter for tax-jurisdiction
promos.

Result: the session sheet is filed and reviewed within 24h; the lead schedules the follow-up charter and provisions a paid test account to unblock the Issue.

SFDPOT - Bach's what-to-vary heuristic

View source (opens in new window)

SFDPOT - Bach's what-to-vary heuristic

Deep reference for exploratory-testing SKILL.md. SFDPOT is James Bach's "San Francisco Depot" heuristic - a catalogue of what can be varied during testing. Published at satisfice.com (opens in new window) and in James Bach's testing-curriculum materials.

If HICCUPPS-F (hiccupps-f.md (opens in new window)) catalogues what to compare against (oracles), SFDPOT catalogues what to change during exploration. Together they form a complete "how to think about a session" pair.

When to use

  • Authoring a charter: pick which SFDPOT dimensions the session should vary (charter-template.md (opens in new window)).
  • Mid-session: tester feels stuck - walk SFDPOT for new ideas of what to vary.
  • Bug-bashing: assign different dimensions to different testers so coverage spreads.

How to use

  1. Name the target. State the feature or area under test in one line (e.g. "the checkout flow").
  2. Walk all six dimensions. For each of S, F, D, P, O, T, ask its prompt from the table below and jot 1-3 concrete variables worth trying. Pull ideas from the full per-dimension catalog below when a dimension feels thin.
  3. Fold the picks into a charter. Each dimension's line becomes a mission for the session - see the umbrella SKILL.md's charter section.
  4. Explore, then re-walk when stuck. If ideas dry up mid-session, walk SFDPOT again - a dimension you skipped usually holds the next bug.
  5. Interpret with an oracle. For each surprise, reach for HICCUPPS-F (hiccupps-f.md (opens in new window)) to decide whether it is actually a bug.

The worked example below applies exactly these steps to a checkout charter.

The six dimensions

SFDPOT names six axes of variation. Pick one or more per session.

DimWhat to varySample variables
S - Structurehow the system is builtcode paths, build options, module topology, cache / buffer state, cluster vs single instance
F - Functionwhat the system doesindividual features, feature combinations, error / recovery paths, undo / redo / rollback
D - Datathe values it handlesboundaries (0, max, max+1), encodings, volumes (empty to 1B), corruption, null / NaN / Infinity
P - Platformthe environment it runs onOS, browser, device, hardware, network (bandwidth / latency), locale
O - Operationshow it is usedworkflows, user goals, novice vs expert pacing, concurrency, frequency
T - Timewhen / how long things happenduration, order (A then B vs B then A), race conditions, clock edges (DST, leap day), TTLs, session expiry

Data varies what values; Operations varies how the user moves through.

The six dimensions in full

S - Structure

What can I vary about how the system is built?

The internals of the system. Includes:

  • Code paths (branches, recursive depths)
  • Build / compiler options
  • Module / component connection topology
  • Internal-data structures (cache state, in-memory buffers, thread pools)
  • Deployment shape (single instance vs cluster, sidecar vs not)

A "Structure" exploration might toggle internal options, route through a non-standard code path, or inspect how the system behaves under a non-default build.

F - Function

What can I vary about what the system does?

The feature surface area. Includes:

  • Functions / features (each can be exercised individually)
  • Feature combinations (feature A + feature B interaction)
  • Error / recovery paths (what happens when X fails?)
  • Boundary / edge functions (cancel, undo, redo, rollback)

A "Function" exploration runs each function - and especially combinations - that the test plan didn't enumerate.

D - Data

What can I vary about the values the system handles?

The input + state space. Includes:

  • Input boundary values (0, 1, max, max+1, min, min-1)
  • Input formats / encodings (UTF-8, UTF-16, Windows-1252)
  • Data volumes (empty, single, 1k, 1M, 1B)
  • Data shapes (deeply nested, flat, sparse, dense)
  • Data corruption (truncated, malformed, missing fields)
  • Special values (null, undefined, NaN, Infinity)

A "Data" exploration feeds pathological inputs - see malicious-payload-bank for canonical payloads.

P - Platform

What can I vary about the environment the system runs on?

The deployment platform. Includes:

  • OS (Windows / Linux / macOS, version)
  • Browser (Chrome / Firefox / Safari / Edge, version)
  • Mobile device (iOS / Android, version, model)
  • Hardware (CPU architecture, memory, storage)
  • Network (Wi-Fi vs cellular, low bandwidth, high latency, lossy)
  • Locale (language, region, timezone, calendar)

A "Platform" exploration tests across the matrix - compose with the browser/OS matrix skills in qa-web-e2e / qa-ci-integration for systematic matrix testing.

O - Operations

What can I vary about how the system is used?

User behaviour patterns. Includes:

  • User workflows (paths through the UI / API)
  • Tasks (the user's actual goals - see ISTQB use-case)
  • User skill levels (novice vs expert pacing, undo + redo frequency)
  • Concurrency (single user vs many, simultaneous edits)
  • Frequency (rare event vs continuous use)

An "Operations" exploration simulates real user workflows rather than test scripts.

T - Time

What can I vary about when / for how long things happen?

Temporal dimensions. Includes:

  • Duration (1 ms, 1 s, 1 min, 1 hour, 1 day, 1 year of uptime)
  • Order (do A then B vs B then A)
  • Concurrency / race conditions (A and B simultaneously)
  • Clock edges (DST transition, leap day, year-end rollover, leap second)
  • Cache TTLs (just expired vs just refreshed)
  • Session timeouts (just before expiry, at expiry, after expiry)

A "Time" exploration is the hardest to plan - many time-related bugs require deliberate clock manipulation.

Worked example - applying SFDPOT to a checkout charter

Charter: "Explore the checkout flow to discover bugs."

Apply SFDPOT to plan the session:

- **S - Structure:** Toggle the `feature-new-checkout=true` flag
  in middle of the session to compare old vs new code paths.
- **F - Function:** Stack multiple promos; combine with gift card;
  combine with store credit. Trigger refund mid-checkout.
- **D - Data:** Cart with 0 items, 1, 100, 1000 items. Cart with
  free items only. Cart with $0.01 total. Cart with $9999.99
  total. Invalid product IDs.
- **P - Platform:** Safari iOS (autofill / Apple Pay), Chrome
  desktop, low-bandwidth Android.
- **O - Operations:** Add, remove, re-add an item. Navigate
  away and return. Refresh during payment processing.
- **T - Time:** Apply a promo that expires in 1 minute, then
  delay 70 seconds. Open two tabs simultaneously and check out
  from each.

This shapes a richer session than "click around the checkout page." Each bullet becomes one mission line in the charter.

SFDPOT vs other heuristics

HeuristicWhat it catalogs
SFDPOTWhat to vary during exploration
HICCUPPS-FWhat to compare against when interpreting observations
FCC-CUTS-VIDSWhat to list about the system (specification-style)
CRUSSPIC-STMPLQuality criteria to evaluate against
Tours (Whittaker)Themed exploration missions

They're complementary, not competing - a strong session uses multiple. Bach + Bolton's Rapid Software Testing curriculum teaches all of them as a toolkit.

Anti-patterns

Anti-patternWhy it failsFix
Picking one dimension onlyOther dimensions yield bugs the chosen one missesMention each dimension's pick in the charter
Skipping TimeTime bugs are common but easy to forgetAlways consider T in any session involving state
Confusing Data + OperationsVariants overlapData = what values; Operations = how the user moves through
Pre-scripting SFDPOT into test casesDefeats explorationSFDPOT is for generating ideas; the tester decides moment-to-moment
Operations as "user persona testing"Not exactly: Operations is variant-of-use, not variant-of-user (which is Platform's locale + skill / familiarity)Treat O as variation of use-patterns

Limitations

  • Heuristic, not exhaustive. Bugs can hide in dimensions SFDPOT doesn't enumerate (cross-cutting concerns like security, observability) - those have their own heuristic catalogs.
  • Requires deliberate practice. A new tester walks SFDPOT mechanically; an experienced one fluently considers each dimension in seconds.
  • Variation cost is real. Some dimensions (Platform full matrix; Time clock manipulation) require infrastructure that may not exist for ad-hoc exploration.

References

Whittaker's seven exploratory tours

View source (opens in new window)

Whittaker's seven exploratory tours

Deep reference for exploratory-testing SKILL.md. Tours keep an exploratory session focused: each tour frames the exploration around a theme that catches a specific class of bugs. The seven canonical tours come from James Whittaker's Exploratory Software Testing (2009, Addison-Wesley). The charter author and the tester pick which tours to apply per session.

For recon tours on a product the tester does not yet know, see fcc-cuts-vids.md (opens in new window) - Kelly's eleven tours precede Whittaker's by four years and serve the earlier lifecycle stage.

How to use

  1. Name the mission. State what the session must learn (new feature, post-refactor regression, bug-cluster hunt, compliance audit).
  2. Choose 1-3 tours. Use the Picking tours per charter table to map the mission to tours. A 90-minute charter fits 1-3 tours, never all seven.
  3. Read each chosen tour's detail below - its mission, signal, worked example, and when-to-use notes.
  4. Run each tour as a lens, not a checklist. Adapt mid-tour as you learn; pair the Bad-data tour with malicious-payload-bank (a canonical starter payload for that step: '; DROP TABLE users; -- to probe SQL-injection escaping) and the Intellectual tour with a domain-expert guide.
  5. Capture findings per tour with the PROOF fields (Capturing tour findings below).
  6. File into the session sheet (session-sheet-and-metrics.md (opens in new window)) and rotate tours, testers, and scope across releases.

The seven tours at a glance

TourMissionSignal it surfaces
FeatureVisit every in-scope feature at depth 1Does the feature exist / open / work?
MoneyFind every place money / pricing / discount appears; verify eachRounding, currency drift, discount-stacking, locale formatting
LandmarkVisit the canonical hero flowsMarquee features still work after a refactor
IntellectualExplore the hardest-to-explain featuresBugs in genuinely complex business logic
Bad-dataFeed pathological inputsValidation gaps, error handling, security, locale parsing
ConfigurationVary user / system configConfig-dependent bugs (flags, theme, locale, browser)
Garbage collector'sVisit every page / endpoint onceDead links, 404s, stale routes, render issues

Picking tours per charter

A 90-minute charter can include 1-3 tours. Pick based on the mission:

MissionRecommended tours
New-feature explorationFeature tour + Money tour (if money) + Bad-data tour
Post-refactor regression checkLandmark tour + Garbage collector's tour
Bug-cluster investigationIntellectual tour + Bad-data tour
Compliance / auditMoney tour + Configuration tour
New tester onboardingFeature tour (alone) + reflection

A charter with all 7 tours is too broad; the tester won't have time to apply any of them well.

Capturing tour findings

Per the PROOF debrief format (debrief.md (opens in new window)), each tour produces:

  • Past: which paths the tour covered.
  • Results: what the tour surfaced (per-finding).
  • Outlook: what the tour didn't reach; recommend follow-up charter scope.
  • Obstacles: what blocked the tour (broken setup, missing test data).
  • Feelings: tester's qualitative read after the tour.

The seven tours in full

Tour 1 - Feature tour

Mission: Visit every feature in scope at depth = 1.

Signal: "Does the feature exist? Does it open without an error? Does its primary affordance work?"

Example application:

**Charter:** Explore the dashboard.
**Feature tour:**
1. Open the dashboard. Pass.
2. Click "Notifications" → notification panel opens. Pass.
3. Click "Settings" → settings page loads. Pass.
4. Click "Reports" → 404. **FAIL** - investigate.

When to use: New feature; post-deploy smoke; feature-coverage gap survey.

When NOT to use: Deep-dive sessions where the depth-1 sweep provides no signal.

Tour 2 - Money tour

Mission: Find every place money / pricing / currency / discount appears; verify each.

Signal: Rounding errors, currency conversion drift, discount-stacking bugs, free-shipping edge cases, locale-specific formatting (€1.234,56 vs $1,234.56).

Example application:

**Charter:** Explore promo code application.
**Money tour:**
1. Apply 10% off promo to a $24.99 cart. Verify subtotal = $22.49.
2. Apply 50% off promo to a $0.01 cart. Verify subtotal = $0.01 (rounding).
3. Apply 100% off promo to a free-shipping order. Verify shipping handling.
4. Apply two stackable promos. Verify the order of operations.
5. Apply a promo + state tax. Verify tax base.

When to use: Any feature touching money, pricing, billing. Critical for: Checkout, billing, subscription management.

Tour 3 - Landmark tour

Mission: Visit each "landmark" feature - the canonical user journeys / hero flows.

Signal: Whether the marquee features still work after a refactor; baseline confidence.

Example application:

**Charter:** Verify post-refactor regression risks.
**Landmark tour:**
1. Sign up new account → confirm email → log in. **Hero flow.**
2. Add to cart → checkout → confirmation. **Hero flow.**
3. Cancel subscription → reactivate. **Hero flow.**

When to use: Post-refactor verification; pre-release smoke; quarterly health check.

When NOT to use: When the team already has automated tests for hero flows (those should run first; tour confirms behavior the automation doesn't catch).

Tour 4 - Intellectual tour

Mission: Explore the hardest-to-understand parts of the product. The features that the team has trouble explaining.

Signal: Bugs in genuinely complex business logic where edge cases lurk.

Example application:

**Charter:** Explore the tax calculator's nexus rules.
**Intellectual tour:**
1. Order ships from CA to OR (no sales tax in OR). Verify tax = 0.
2. Order ships from CA to TX (Texas nexus). Verify TX tax applied.
3. Order ships from CA to NY (origin-based vs destination-based). Verify rule.
4. Order with mixed-tax-rate items. Verify per-item rate application.
5. Subscription order spanning a tax-rate change date. Verify proration.

When to use: Complex business logic (tax, billing, permissioning, scheduling).

Effort: High. This tour requires the tester to understand the domain - pair with a domain-expert "guide."

Tour 5 - Bad-data tour

Mission: Feed pathological inputs and observe behavior.

Signal: Input validation bugs, error-handling gaps, security vulnerabilities, locale parsing issues.

Example application:

**Charter:** Stress-test the search input.
**Bad-data tour:**
1. Empty input. Verify behavior.
2. Single space. Verify trimming or rejection.
3. 5000-character input. Verify truncation or rejection.
4. SQL injection: `'; DROP TABLE users; --`. Verify escaping.
5. XSS: `<script>alert(1)</script>`. Verify sanitization.
6. Unicode bidi override (RLO): `‮`. Verify handling.
7. Right-to-left text: `مرحبا`. Verify rendering.
8. Emoji + ZWJ sequences: `👨‍👩‍👧‍👦`. Verify counting.
9. Null byte: `foo\0bar`. Verify handling.

Pair with malicious-payload-bank for the canonical payloads (OWASP Top 10 + CWE Top 25).

When to use: Any input field (search, forms, URL params, file upload).

Tour 6 - Configuration tour

Mission: Vary the user's / system's configuration; observe behavior changes.

Signal: Config-dependent bugs (feature flags off vs on, dark mode vs light, locale variations, browser variations).

Example application:

**Charter:** Verify checkout works under all account configurations.
**Configuration tour:**
1. New user, no payment method. Verify "add payment" prompt.
2. Existing user, expired card. Verify "update card" prompt.
3. EU user, GDPR consent banner active. Verify checkout flow.
4. Beta user with experiment flag `new-checkout=true`. Verify variant.
5. Admin impersonating a user. Verify behavior.

When to use: Multi-tenant / multi-config products; before toggling a major feature flag.

Pair with feature-flag-test-harness for the matrix-shard approach to flag-combination testing.

Tour 7 - Garbage collector's tour

Mission: Visit every page / endpoint once. Don't deeply test; just confirm presence.

Signal: Dead links, 404s, stale routes, removed-feature breadcrumbs.

Example application:

**Charter:** Pre-release sanity check.
**Garbage collector's tour:**
1. Walk through every nav item; confirm each loads.
2. Visit every footer link; confirm each loads.
3. Visit every URL listed in the sitemap; flag 404s.
4. Visit every documentation link from the in-app help.

When to use: Before a release; after a major refactor; periodic health check.

When NOT to use: Replacing automated link-checking - the garbage collector's tour is for rendering issues an automated checker can't catch.

Worked example - a new promo-checkout charter

Charter: "Explore the new promo-code checkout with sample carts to discover discount + input bugs."

1. Mission: new-feature exploration.
2. Pick tours (from the table): Feature + Money (money present) + Bad-data.
3. **Feature tour:** open checkout; promo field renders; apply button works. Pass.
4. **Money tour:** apply 10% off to a $24.99 cart → expect $22.49. Then
   stack two promos and check order of operations. FOUND: "STACK50"
   applies after tax instead of before, reproduces 3/3.
5. **Bad-data tour:** paste a 5000-char code → expect rejection;
   `'; DROP TABLE users; --` → expect escaping. Both handled.

Result: one confirmed bug (B-001, promo applied post-tax) logged to the session sheet; Outlook recommends a follow-up Configuration-tour charter for EU-VAT carts.

Tour anti-patterns

Anti-patternWhy it failsFix
Picking all 7 tours for one sessionTester touches each superficially; no depth.1-3 tours per 90-min session.
Money tour without monetary fieldsThe tour wastes time on "verify nothing changed."Pick tours per the feature; not all features need every tour.
Garbage collector's tour without a sitemapTester misses pages; coverage gaps invisible.Use the team's sitemap / docs as the seed list.
Treating a tour as a checklistTour is a heuristic; rigid stepwise application defeats the exploration.Tester adapts mid-tour as they learn (per the exploratory definition).
Bad-data tour with random inputsRandom isn't useful; structured pathological inputs are.Use canonical payloads (malicious-payload-bank).
Intellectual tour without a domain expert pairTester misses the actual complexity; tour is shallow.Pair with someone who knows the domain.
One tour run per release without rotationThe same tour by the same tester catches the same bugs (or none).Rotate which tours run, which testers, what scope (Picking section).

Limitations

  • Effectiveness varies by tester skill. Seniors apply tours flexibly; juniors may fall back to using them as checklists.
  • Tour vocabulary is community-conventional. Whittaker's seven are the most widely cited, but some teams use additional tours (Performance tour, Security tour, Accessibility tour) - note that those overlap with dedicated specialty plugins.
  • No automated tour. Tours are by definition human-driven. For automated equivalents, see the named-tool skills in other plugins.

References

Related skills

bug-bash-facilitator

Builds a structured bug-bash session - pre-bash kit (charter, test-data prep, environment setup, sign-up sheet), in-bash structure (role rotation across cohorts, shared backlog board, real-time triage), scoring rubric (severity weighting, novelty bonus), and a post-bash same-day wrap-up authored by the facilitator (not a standalone debrief: for post-session writeups without a live bash, use the PROOF debrief in exploratory-testing). Use when a team needs a coordinated multi-tester sweep before a release or after a major change - converts an ad-hoc "everyone test for an hour" into a recorded, comparable session with deliverables.

decision-table-test-design

Derives human-readable manual test cases from a business-rule spec via a decision table: identify conditions and actions, build the full 2^n-column matrix, collapse columns with irrelevant entries, strike infeasible combinations, then emit one test case per remaining column (each feasible column is one coverage item per ISTQB CTFL v4.0 section 4.2.3). A deep single-technique walkthrough rather than a broad multi-lens case matrix; the output is manual step/expected cases rather than parameterized test code, and it covers how cases are derived rather than how a case record is structured. Use when a spec's outcome depends on interacting conditions (pricing, eligibility, discounts, routing rules) rather than the boundaries of a single input.

exploratory-charter-author

Authoring workflow that turns a feature spec, risk area, or bug cluster into a session-based exploratory testing charter per Jonathan and James Bach's SBTM - frames the one-sentence mission, scopes 3-7 areas, picks a 60 / 90 / 120 min time-box, suggests tours, and wires the PROOF debrief deliverables. Per Bach, exploratory testing is "performing tests while learning things that may influence the testing" - the charter sets the mission while leaving exact steps to the tester's judgment. Use when a feature has too many unknowns to script (new feature / refactor blast-radius / bug cluster) and a session-based exploration is the right approach. Authors the charter only: the ready-to-fill charter card, session vocabulary, debrief template, and session review live in the exploratory-testing skill this workflow composes with.

manual-test-script-author

Builds stakeholder-readable scripted manual test cases from a feature spec in four formats: a step-table (preconditions / steps / expected result / actual / pass-fail / notes) for spreadsheet review, a Gherkin Given/When/Then format for BDD-aware teams, a business-language UAT script with acceptance-criteria mapping and contractual sign-off (references/uat-format.md), and a one-line-per-item execution checklist for smoke / on-call / bug-bash / compliance sweeps (references/checklist-format.md). Each script is self-contained (no implicit team knowledge), single-scenario (one happy + N edge per script), and includes the data setup the tester needs without being a developer. Use when a feature can't be (or shouldn't be) fully automated and a human tester needs an executable script or checklist - UAT sign-off rounds, regression baselines, certification testing, deploy smoke checklists, exploratory follow-up scripts.

state-transition-test-design

Derives human-readable manual test cases from stateful behavior: identify states, events, transitions, and guard conditions, draw the state table including invalid (empty-cell) transitions, choose a coverage level (all states, valid transitions / 0-switch, transition pairs / 1-switch per Chow, all transitions including invalid ones), then derive one test case per coverage item as an event sequence with per-step expected states (ISTQB CTFL v4.0 section 4.2.4). A deep single-technique walkthrough rather than a broad multi-lens case matrix; the output is manual step/expected cases rather than parameterized test code, and it covers how cases are derived rather than how a case record is structured. Use for lifecycle entities (accounts, orders, subscriptions), workflows, and UI wizards where the response to an event depends on the current state.