living-documentation-publisher

Overview

Per the Serenity BDD book (serenity-bdd/the-serenity-book, living-documentation.adoc):

"Living documentation is generated by the automated test suite, and is therefore by definition it is always up-to-date."

The same source distinguishes living documentation from plain test reports: living documentation is authored before development starts, targets the whole team (BAs, product owners, stakeholders), and reads as a business-functionality narrative, not a pass/fail table.

This skill covers the full workflow: produce Cucumber JSON from a passing suite, feed it into a report renderer, apply tag-based section routing, and publish the HTML artefact to a Pages endpoint so stakeholders get a URL, not a zip file.

Two primary renderers are covered:

• multiple-cucumber-html-reporter (Node, any Cucumber-JS project)
• Serenity BDD aggregate report (JVM, Maven/Gradle projects using CucumberWithSerenity)

When to use

• The team already has Cucumber scenarios (see cucumber-testing) and stakeholders need to read acceptance-criteria status without opening a test runner.
• The living-documentation page must track only passing scenarios (not every scenario authored, including pending or skipped ones).
• CI should gate report publication so stakeholders never see a stale or red-run document.

If only engineers read the results, the JUnit XML feed to junit-xml-analysis is sufficient.

Step 1 - Produce Cucumber JSON output

The report renderers both consume Cucumber JSON. Configure the JSON formatter before wiring the renderer.

Cucumber-JS (in package.json):

{
  "scripts": {
    "test": "cucumber-js features/ --format json:reports/cucumber.json"
  }
}

Use a timestamped filename when running parallel shards to avoid overwrite (multiple-cucumber-html-reporter usage):

cucumber-js features/ \
  --format json:reports/cucumber-$(date +%s).json

Cucumber-JVM with Serenity (in serenity.properties):

The CucumberWithSerenity runner captures results automatically; no explicit JSON formatter is needed. Serenity writes its own JSON artefacts to target/site/serenity/ as part of mvn verify (serenity-bdd/the-serenity-book, maven.adoc).

Step 2 - Generate the HTML report (Node path)

Install the reporter as a dev dependency (multiple-cucumber-html-reporter installation):

npm install multiple-cucumber-html-reporter --save-dev

Create scripts/generate-report.js:

const report = require("multiple-cucumber-html-reporter");

report.generate({
  // required
  jsonDir: "./reports/",
  reportPath: "./docs/living-documentation/",

  // identification metadata shown in the report header
  metadata: {
    browser: { name: "chrome", version: "latest" },
    device: "CI runner",
    platform: { name: "linux", version: "22.04" }
  },

  // custom info block (release, project, branch)
  customData: {
    title: "Run info",
    data: [
      { label: "Project", value: "Checkout Service" },
      { label: "Release", value: process.env.RELEASE_TAG || "dev" }
    ]
  },

  // display options
  reportName:      "Checkout Service - Living Documentation",
  pageTitle:       "Acceptance Criteria Status",
  displayDuration: true,
  durationInMS:    true
});

Run it after the test step (multiple-cucumber-html-reporter usage):

npm test && node scripts/generate-report.js

Key options from the official docs (multiple-cucumber-html-reporter options):

Step 3 - Generate the HTML report (JVM / Serenity path)

Add the Serenity Maven plugin and bind it to post-integration-test (serenity-bdd/the-serenity-book, maven.adoc):

<plugin>
  <groupId>net.serenity-bdd.maven.plugins</groupId>
  <artifactId>serenity-maven-plugin</artifactId>
  <version>${serenity.maven.version}</version>
  <executions>
    <execution>
      <id>serenity-reports</id>
      <phase>post-integration-test</phase>
      <goals><goal>aggregate</goal></goals>
    </execution>
  </executions>
</plugin>

Run the full pipeline:

mvn verify

Or regenerate the report from existing test data without re-running tests:

mvn serenity:aggregate

The Requirements tab of the generated report renders living documentation: Serenity reads the directory hierarchy under src/test/resources/features/[theme]/[capability]/ and maps it to the requirements hierarchy (serenity-bdd/the-serenity-book, living-documentation.adoc).

Set hierarchy labels in serenity.properties:

serenity.requirements.types=theme,capability,story

Add a readme.md at each directory level; Serenity renders it as contextual prose above the scenario list, turning the Requirements tab into a readable illustrated user manual (serenity-bdd/the-serenity-book, living-documentation.adoc).

Step 4 - Tag scenarios for report sections

Use Gherkin tags to categorise scenarios in both renderers.

In the feature file:

@billing @sprint-12
Scenario: Apply promo at checkout
  Given ...

Serenity tag filtering - run only tagged tests and limit the aggregate report to those requirements (serenity-bdd/the-serenity-book, filtering-reports.adoc):

# Run and report on one sprint
mvn clean verify -Dcucumber.options="--tags=@sprint-12" \
  -Dtags=sprint-12

# Post-run report filtered to a tag
mvn serenity:aggregate -Dtags=sprint-12

Note: "requirements filtering only happens if you specify the tags option" (serenity-bdd/the-serenity-book, filtering-reports.adoc).

Excluding pending/WIP from published docs:

# Cucumber-JS: exclude anything tagged @wip from the JSON
npx cucumber-js features/ \
  --tags "not @wip" \
  --format json:reports/cucumber.json

This keeps the living-documentation page free of scenarios that are not yet passing.

Step 5 - Only publish passing runs

Gate the report publication step on a green test exit code. In a shell script:

set -e
npm test                         # exits non-zero on any failure
node scripts/generate-report.js  # only reached if all tests pass

For Serenity, use serenity:check after verify (serenity-bdd/the-serenity-book, maven.adoc):

mvn verify serenity:check        # fails the build if any scenario is red

This ensures the published artefact reflects only a fully-green run.

Step 6 - Publish to GitHub Pages (CI)

GitHub Actions

name: Living Documentation

on:
  push:
    branches: [main]

jobs:
  publish-docs:
    runs-on: ubuntu-latest
    permissions:
      contents: write
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: 20

      - name: Run tests and generate report
        run: |
          npm ci
          npm test
          node scripts/generate-report.js

      - name: Publish to GitHub Pages
        uses: peaceiris/actions-gh-pages@v4
        with:
          github_token: ${{ secrets.GITHUB_TOKEN }}
          publish_dir: ./docs/living-documentation

For Serenity (JVM), replace the generate step and point publish_dir at target/site/serenity.

GitLab Pages

pages:
  stage: deploy
  script:
    - npm ci
    - npm test
    - node scripts/generate-report.js
    - mkdir -p public
    - cp -r docs/living-documentation/* public/
  artifacts:
    paths:
      - public
  only:
    - main

See github-actions-test-jobs for general CI test-job conventions.

Anti-patterns

Limitations

• Serenity aggregate requires JVM test execution in the same Maven lifecycle. Running serenity:aggregate against JSON files from a Cucumber-JS run is not supported; use multiple-cucumber-html-reporter for Node projects.
• multiple-cucumber-html-reporter merges JSON by filename, not by run order. If parallel shards write JSON with identical names, results are silently overwritten. Use timestamped or shard-indexed filenames.
• Pages publish latency. GitHub Pages can take 60-120 seconds to reflect a push; stakeholders may see the previous version briefly after a CI run.
• Serenity requirements hierarchy is derived from directory layout. Renaming a feature directory breaks historical trend data in the report.

References

• ld - Serenity BDD book, living-documentation.adoc: definition, Requirements tab, hierarchy, serenity.properties keys (serenity.requirements.types, report.assets.directory), readme.md enrichment, evidence API.
• mv - Serenity BDD book, maven.adoc: serenity-maven-plugin coordinates (net.serenity-bdd.maven.plugins:serenity-maven-plugin), aggregate goal, post-integration-test phase binding, mvn verify, serenity:check.
• fr - Serenity BDD book, filtering-reports.adoc: -Dtags, -Dcucumber.options, requirements-filtering caveat.
• mchr-install - multiple-cucumber-html-reporter installation: npm/yarn/pnpm, CucumberJS version compatibility matrix.
• mchr-usage - multiple-cucumber-html-reporter usage: AfterFeatures hook (CucumberJS 2.x) vs. separate post-test script (3.x+), timestamped JSON filenames.
• mchr-opts - multiple-cucumber-html-reporter options: jsonDir, reportPath, reportName, pageTitle, displayDuration, durationInMS, saveCollectedJSON, customStyle, overrideStyle full option table.
• cucumber-testing - upstream skill: produce Cucumber JSON with --format json:.
• junit-xml-analysis - engineer-facing test result analysis (separate concern from stakeholder docs).
• github-actions-test-jobs - CI job conventions for the publish step.