qa-search-relevance
Search relevance testing for Elasticsearch, OpenSearch, Solr, vector and hybrid search: judgment lists and NDCG, MRR and recall@k regression checks. Builds judgment lists with rater guidelines, benchmarks vector recall against latency, and measures fusion lift in hybrid pipelines. A review agent (relevance-regression-reviewer) compares before and after metrics per query.
Install this plugin
View source (opens in a new tab)/plugin install qa-search-relevance@testland-qaFirst time? Add the marketplace with /plugin marketplace add testland/qa
Plugin
- Category
- AI & ML
- Version
v1.3.4- Contents
- 4 skills · 1 agent
- Part of
- qa-role-ai
Skills4
elasticsearch-relevance-testsUse before changing analyzers, synonyms, boosts or query templates on Elasticsearch, OpenSearch or Solr search, so a relevance baseline exists.skillhybrid-search-eval-authorUse when a search system combines keyword (BM25) and vector retrieval, with or without a reranker, and needs numbers: judgment sets, nDCG@k and MRR on fused results, Reciprocal Rank Fusion vs weighted fusion vs single-stage, and whether the reranker earns its latency.skilljudgment-list-authorUse when a team must create or refresh the human relevance judgments that search tests rely on, before running NDCG, MRR or recall evaluations: query sets, grading scales, rater guidelines, inter-rater agreement, Quepid, TREC-style pooling and refresh cadence.skillvector-search-recall-testsUse when HNSW or IVF parameters are being tuned, an embedding model is swapped, or vector search recall was never measured.skill
Agents1
About qa-search-relevance
IR-metrics-driven search relevance testing - judgment lists, NDCG / MRR / Recall@k, vector recall@k vs latency Pareto curve. Four skills + one reviewer agent that synthesizes per-query regression analysis across term-based and vector search.