qa-data-notebooks
Jupyter notebook testing: 4 skills (nbval-tests, notebook-ci-pipeline-author, papermill-tests, testbook-tests) and 1 agent (notebook-quality-reviewer). Covers full-notebook regression (nbval), function-level unit tests (testbook), and parameterized execution (papermill).
Install this plugin
/plugin install qa-data-notebooks@testland-qaPart of role bundle: qa-role-ai
qa-data-notebooks
Jupyter notebook testing - three complementary tools, one workflow. Use papermill as the executor (parameterize + run), nbval for full-notebook output regression, testbook for function-level unit tests against notebook-defined functions.
Components
| Type | Name | Description |
|---|---|---|
| Skill | nbval-tests | pytest --nbval regression: re-run cells, compare to stored output; --nbval-lax for tutorials; per-cell controls (#NBVAL_SKIP, etc.); sanitize regex for dynamic outputs |
| Skill | testbook-tests | @testbook decorator + tb.ref() / tb.inject() / tb.patch() for function-level unit tests; pytest fixture pattern for shared kernel |
| Skill | papermill-tests | Parameterized execution (CLI + Python API); parameters cell tag; matrix sweeps; pairs with nbval/testbook |
| Agent | notebook-quality-reviewer | Adversarial PR reviewer: flags untested cells, --nbval-lax misuse, hardcoded credentials, non-deterministic outputs, missing parameters tag, and committed outputs; emits BLOCK/PASS verdict |
| Skill | notebook-ci-pipeline-author | Stand up the full notebook CI pipeline: papermill execute, nbval regression, testbook unit, artifacts. |
Install
/plugin marketplace add testland/qa
/plugin install qa-data-notebooks@testland-qaSkills
nbval-tests
Validate Jupyter notebooks via the `pytest --nbval` plugin - re-execute cells and compare outputs to stored results. Cover the strict path (output match required), `--nbval-lax` (failure-only), `--sanitize-with` for dynamic outputs, and per-cell controls (`#NBVAL_SKIP`, `#NBVAL_IGNORE_OUTPUT`, `#NBVAL_RAISES_EXCEPTION`). Use when a repo ships `.ipynb` files as tutorials, docs, or analyses that must keep producing the same outputs after a dependency upgrade or source change.
notebook-ci-pipeline-author
Wires the papermill-tests, nbval-tests, and testbook-tests skills into a single working GitHub Actions CI pipeline: parameterized execution (papermill) -> output regression (nbval) -> function unit tests (testbook) -> artifact upload (executed .ipynb + HTML report). Use when a team has notebook tests spread across the three tools but assembles the pipeline manually and needs a single authoritative workflow file with output stripping (nbstripout), pip caching, and structured failure reporting.
papermill-tests
Use Papermill to parameterize and execute notebooks in CI as regression tests - `papermill input.ipynb output.ipynb -p alpha 0.6` (CLI) or `pm.execute_notebook(...)` (Python API). Use when notebooks must run as parameterized regression jobs in CI.
testbook-tests
Use the `@testbook` decorator to write conventional pytest unit tests against functions defined in Jupyter notebooks, without copy-pasting the function into a `.py` file. Covers `tb.ref()` (notebook object access) and `tb.inject()` (insert code into the kernel) for hermetic per-test setup. Use when unit-testing functions that live in a Jupyter notebook.