qa-ml-models
ML model testing: Giskard and Deepchecks validation, Evidently drift monitoring, Fairlearn fairness metrics, and model performance regression gates. Assigns risk tiers with the fairness and explainability evidence each tier requires, and wires Jupyter notebook CI with papermill, nbval, testbook and nbstripout.
/plugin install qa-ml-models@testland-qaFirst time? Add the marketplace with /plugin marketplace add testland/qa
- Category
- AI & ML
- Version
v1.4.4- Contents
- 7 skills
- Part of
- qa-role-ai
Skills7
deepchecks-testsUse before training to catch train-test leakage and data integrity problems in tabular, NLP or vision data, and to rerun the same suite on production samples for drift.skillevidently-monitoringUse when a tabular ML model needs a drift or quality gate in CI, a scheduled monitoring job, or triage of a fired drift alert.skillfairlearn-fairnessUse when a model's decisions affect people and an auditor or rule (ECOA, GDPR Art. 22, EU AI Act) needs per-group evidence, or someone reports unfair treatment of a group.skillgiskard-testsUse when a trained tabular or NLP model is about to ship with no tests of its own, or a feature or hyperparameter change needs a pre-merge check.skillmodel-performance-regression-gateUse when a retrained ML model is a candidate for promotion and CI must block it if it got worse.skillmodel-risk-evidence-matrixUse when an ML model release is up for promotion and someone must decide which fairness and explainability evidence is required, or check the evidence against the model's risk tier.skillnotebook-ci-pipeline-authorUse when Jupyter notebooks must run in CI as tests, their outputs must stay stable, or a PR adding notebooks needs review.skill
About qa-ml-models
ML model testing: vulnerability scanning, data validation, drift monitoring with alert triage, group fairness, and risk-tiered evidence gating. Seven skills covering Giskard (scan() + test catalog), Deepchecks (suites for data integrity / train-test / model evaluation), Evidently (drift monitoring + drift-alert triage playbook), Fairlearn (MetricFrame + Reductions mitigation), a retrain regression gate, the model-risk evidence matrix (risk tiering, fairness gating workflow, and Alibi Explain per-prediction explanation records), and the Jupyter notebook CI pipeline (papermill + nbval + testbook + nbstripout).