Changelog #44

Accessibility and Code Quality Test Suite

The UCSB Web Theme accessibility and code quality test suite helps maintainers, site owners, and developers identify issues in upstream and site-specific Drupal code.

The suite runs automatically in GitHub Actions on upstream pull requests and can also be run locally through Drush. It checks custom modules, themes, subthemes, and other site-specific code so issues can be identified and remediated earlier in the development process.

What the Test Suite Includes

GitHub Actions support for automated accessibility and code quality testing.
Drush commands for running the same checks locally.
Linting for custom PHP, JavaScript, CSS, Twig, YAML, Markdown, Composer, and GitHub Actions workflow files.
Accessibility checks using Alfa, axe-core, pa11y, reflow, and meta viewport audits.
PHPUnit support for custom Unit and Kernel tests.
Unified HTML reports under:
```
web/sites/default/files/test-reports/
```

Accessibility Checks

Each accessibility check has its own Drush command, allowing a single check to be run or re-run independently:

drush utest:alfa
drush utest:axe
drush utest:axe-watcher
drush utest:pa11y
drush utest:reflow
drush utest:meta-viewport

Accessibility checks crawl pages from the site's sitemap. By default, each check scans up to 50 pages. Use --max-pages=200 for a deeper scan or --max-pages=all to scan the full sitemap.

An accessibility profile can also be selected for each run. For example, use --a11y-profile=standard to check WCAG 2.0 and 2.1 Level A and AA requirements only. The default profile is comprehensive.

How Results Are Reported

Only critical or serious accessibility findings cause a check to fail. Moderate and minor findings are advisory and remain available in the reports without blocking the check.

After drush utest:all finishes, the suite displays a summary table showing each check as:

PASSED
PASSED (advisory)
FAILED
INCOMPLETE

Pull requests display the same summary. A check is reported as INCOMPLETE when it could not verify everything, such as when one or more pages fail to load during a crawl.

Each check produces its own HTML report, including PHPUnit and pa11y. The unified report links to the individual reports.

Local Setup

After pulling the latest upstream changes, run the first-time setup commands:

drush utest:js-install
drush utest:browsers
drush utest:check-config

Before running accessibility tests, set BASE_URL to the local site you want to test:

export BASE_URL=https://your-site.ddev.site

Use the correct local URL for your environment, such as a DDEV, Lando, or other local development URL.

Site-specific spell-check words are stored in a file that is not tracked by the upstream, so upstream updates do not overwrite them.

Run the Test Suite

Run all tests against the configured local site:

drush utest:all

Run only code quality checks:

drush utest:lint

Run an individual accessibility check:

drush utest:alfa
drush utest:axe
drush utest:axe-watcher
drush utest:pa11y
drush utest:reflow
drush utest:meta-viewport

Run Checks for Specific Components

Code quality and PHPUnit checks can be limited to one or several components for faster local feedback. Multiple modules, themes, or profiles can be listed using commas, and the options can be combined in the same run.

drush utest:lint --modules=module_one,module_two
drush utest:lint --modules=module_one --themes=mysubtheme
drush utest:lint --themes=mysubtheme
drush utest:phpunit --modules=module_one,module_two

Reports

After a test run, start with the unified report:

web/sites/default/files/test-reports/index.html

Each check also writes its own report within the test reports directory. Examples include:

alfa-full/alfa-full-report.html
phpunit/phpunit-report.html

Who Should Use It

Upstream maintainers use the suite to review and remediate issues found in upstream code. Individual site owners and developers can run the suite locally to identify issues in custom modules, custom themes, subthemes, and other site-specific code.

The test suite is intended to identify issues early. Findings in upstream code are handled by upstream maintainers. Findings in site-specific custom code should be reviewed and remediated by the site owner or developer responsible for that code.