Performance Audit

warm_cache tells you how quickly the server answered. performance_audit tells you what a visitor experiences. It runs Lighthouse CI (@lhci/cli) against a few pages and reports LCP, CLS, TBT, FCP, Speed Index and page weight. Run it after the cache warm, so it measures cached pages, which is what visitors get. Each page is loaded anonymously with no query string, so the audit neither bypasses nor splits the cache.

It runs from the template's pipeline, after a deploy. The load test tells you how much traffic the site can take; the audit tells you what a single visitor experiences.

Running it

The audit is manual. It is available by default, but it never runs by itself.

• GitLab: after a deploy, press play on the performance audit review, performance audit staging or performance audit production job. Each one waits for its deploy job, and for its warm job when warming is on. Set CWA_CI_PERFORMANCE_AUDIT_REVIEW, CWA_CI_PERFORMANCE_AUDIT_STAGING or CWA_CI_PERFORMANCE_AUDIT_PRODUCTION to "false" to remove that job. performance audit canary is opt-in (CWA_CI_PERFORMANCE_AUDIT_CANARY="true"), because canary shares production's hostname and would measure whichever pod answers.
• GitHub Actions: run the Performance audit workflow (performance-audit.yml) from the Actions tab. Choose the environment (production, staging or review) and the form factors. For a review app, run the workflow from that branch, because the URL is built from the branch name. Set a repository variable CWA_CI_PERFORMANCE_AUDIT_PRODUCTION, CWA_CI_PERFORMANCE_AUDIT_STAGING or CWA_CI_PERFORMANCE_AUDIT_REVIEW to "false" to disable that environment.

By default it audits the first three pages of /sitemap.xml, on mobile, five runs each. That is 15 page loads. Five runs stop one slow run on a shared CI runner from changing the median. Pages are audited one after another, never in parallel, because parallel runs share the runner's CPU and network and skew every metric.

The audit uses real throttling (Lighthouse's devtools method) by default. Lighthouse's simulated throttling gave very different scores for the same page on shared CI runners, from 0.60 to 0.96. Only the performance category is collected.

Settings

Which pages, how many runs, the form factors, throttling and the config file are all CI variables. They're listed in CI/CD Variables → Performance audit.

Budgets

The budgets are in bin/devops/lighthouserc.json. They use Lighthouse's "good" thresholds:

To change them, edit bin/devops/lighthouserc.json, or point CWA_CI_PERFORMANCE_AUDIT_CONFIG at your own file.

A missed error budget fails the audit, but it never fails a deploy. On GitLab the job is allowed to fail, so the pipeline shows "passed with warnings". On GitHub the audit run fails with a warning. That run is only the audit, and the deploy has already finished.

Reading the results

The reports go in performance-report/: Lighthouse's HTML and JSON for each page and form factor, the budget results, and a summary.md table.

The end of the job log shows the same results as an aligned text table. It has one row per page and form factor, from the median run. The columns are Page, Device, Score, LCP, FCP, TBT, CLS, Speed Index and Weight. Each value with a budget is marked:

• ✓ within budget
• ✗ over an error budget, which fails the audit
• ⚠ over a warning budget

The marks are read from the budgets in lighthouserc.json, so they match what failed the audit.

For what a low mobile score usually means, and what to check first, see Reading Lighthouse Scores.

• GitLab: download the job's performance-report/ artifact, kept for 30 days. The job also writes a browser_performance report. GitLab uses it to compare metrics between pipelines in the merge request, but that widget needs GitLab Premium. On Free, you still get the artifact.
• GitHub Actions: the table is in the workflow run's summary. The full reports are in the performance-report-<environment> artifact, kept for 30 days.