Automating Accessibility Checks with axe-core and Playwright

Automating accessibility checks means running a rules engine, here axe-core, against every built page in a real browser and failing the build when it reports a violation. Playwright supplies the browser, and the whole setup is a script of about twenty lines. It found two real problems on our own site. This note shows the setup, reproduces both findings with numbers you can re-run, and lists where the check goes blind.

Versions used for every output below: axe-core 4.13.0, playwright-core 1.63.0, Chromium 141, Node 22.

How do you run axe-core in Playwright?

We load a built page in a Playwright browser, inject axe-core into it, and run it with the wcag2a and wcag2aa tags, which covers the WCAG A and AA rules:

import { createRequire } from 'node:module';
const require = createRequire(import.meta.url);

await page.addScriptTag({ path: require.resolve('axe-core/axe.min.js') });
const results = await page.evaluate(() =>
  axe.run(document, { runOnly: { type: 'tag', values: ['wcag2a', 'wcag2aa'] } })
);

results.violations is the list to act on. Each entry has a rule id, a description, and the nodes that failed. Fail the test if the list is not empty, and print the rule ids and selectors so the failure message is the to-do list:

for (const v of violations) for (const n of v.nodes) console.log(`${v.id}\t${n.target.join(' ')}`);
process.exitCode = violations.length ? 1 : 0;

Three habits made the check more useful than a one-off audit.

Run it on the real built pages. Not on a component in isolation, and not on a dev server with different markup. The built output is what visitors get.

Run it at phone and desktop widths. Layout changes what is visible, and what is visible changes what can fail.

Run it with interactive states open. A closed menu hides its links from the check. Open the menu, then run axe, so the panel is part of the page being tested. The failure modes section below shows this with real output.

What does the pipeline look like?

build site  ->  serve dist/  ->  Playwright opens page (phone + desktop width)
                                      |
                       open menus / dialogs / tabs
                                      |
                   inject axe.min.js, axe.run(wcag2a, wcag2aa)
                                      |
               violations? -- yes --> print "rule<TAB>selector", exit 1
                    |
                    no --> exit 0, then a person still tests by keyboard

Finding one: why did one color token fail 34 elements?

The first finding was contrast. Our dim secondary text color token, #646b76, measured about 3.4:1 against our dark surfaces. The AA minimum for normal-size text is 4.5:1, so it failed.

Contrast ratio is not a matter of taste. It is computed from the relative luminance of the two colors, as (L1 + 0.05) / (L2 + 0.05) with the lighter color on top. We reproduced the finding in a small demo page with that token on the #121418 surface (illustrative page, not our production markup). axe reports:

== before (axe 4.13.0) ==
violations: 2
- color-contrast [serious] nodes=4
     .card:nth-child(2) > .dim:nth-child(1) | Element has insufficient color contrast of 3.42 (foreground color: #646b76, background color: #121418, font size: 12.0pt (16px), font weight: normal). Expected contrast ratio of 4.5:1
- scrollable-region-focusable [serious] nodes=1
     .log | Element should have focusable content
== after (axe 4.13.0) ==
violations: 0

The useful part is where the fix went. The failing color was a token used across the site, so axe reported 34 separate elements on our real pages. We did not touch any of them. When we added 30 more elements using the same class to the demo, axe reported color-contrast: 34 nodes, the same shape of result. We changed the token once, to #7d8590, and checked it against all three surfaces we use. We computed the ratios with the formula above:

Token	on #0b0c0e	on #121418	on #191c21
#646b76 (before)	3.64	3.43	3.18
#7d8590 (after)	5.25	4.94	4.58

The new token clears 4.5:1 on each surface, with the raised surface the tightest at 4.58. (axe printed 3.42 where we round 3.4299 up to 3.43, because axe floors to two decimals; quote the ratio from the tool you gate on.) One edit, 34 failures gone.

If you design with tokens, this is the payoff. A contrast failure is rarely 34 problems; it is usually one wrong value referenced 34 times. Check the token against the darkest and the lightest surface it sits on, because the tightest pairing decides.

Finding two: why could nobody scroll the chat transcript?

The second finding was the chat transcript. It is a scrollable container, and it could not be reached with the keyboard. axe reports this as scrollable-region-focusable, as in the output above. A mouse user can scroll the box; someone navigating by keyboard has no way to focus it and scroll its contents.

The fix was tabindex="0" on the container, plus a role and an accessible label so a screen reader announces what has just received focus:

<div class="log" tabindex="0" role="region" aria-label="Transcript">...</div>

In the demo, adding those three attributes took the rule from one node to zero. It is a small attribute change, but we would not have noticed it ourselves, because we scroll with a mouse or trackpad.

Failure modes we reproduced

A closed menu hides its links. We built a page with a display: none nav panel whose links use the failing token, toggled by a button. We ran axe twice, once before and once after the click:

menu closed : none
menu open   : color-contrast x2

Same page, same rules, a different answer. If the panel is not open when axe runs, the panel is untested. This is the reason our menu test opens the menu first; see building an accessible mobile menu without a framework.

The gate passes silently if it prints nothing. A check that exits 0 on an empty page, or that crashes before axe.run, looks like success. Assert that the page title or a known heading exists before running axe, so a 404 page does not pass.

Tag choice changes the result. With runOnly set to wcag2a and wcag2aa, best-practice rules do not run. That is deliberate for a gate, since those rules are advice, not conformance. Run them in a separate, non-blocking job if you want them.

What doesn’t it catch?

An automated check finds a subset of accessibility issues. It can tell you a color fails a ratio or an element lacks a label. It cannot tell you whether the label is helpful, whether the focus order makes sense, whether a flow is understandable with a screen reader, or whether a custom widget behaves the way a person expects.

So a clean axe run is a floor. It is not a pass. Keyboard-only and screen-reader testing still have to happen by a person, and the automated run’s job is to stop known classes of regression from coming back. If you are reading this to decide whether to skip manual testing, don’t.

Checklist

• Inject axe-core into Playwright and run it with the wcag2a and wcag2aa tags.
• Test the built pages, at phone and desktop widths.
• Open menus, dialogs, and other states before running the check.
• When contrast fails, fix the token, then re-check every surface it sits on.
• Fail on any violation, and print rule ids and selectors.
• Keep keyboard and screen-reader testing in the process.

Related reading

• Building an accessible mobile menu without a framework
• Design systems for small teams
• Speed is a product feature

Frequently asked questions

How do I run axe-core in Playwright?

Load the built page, inject axe-core with page.addScriptTag using the path from require.resolve('axe-core/axe.min.js'), then call axe.run inside page.evaluate with runOnly set to the wcag2a and wcag2aa tags. Read results.violations, print each rule id and selector, and exit non-zero if the list is not empty.

Why does axe-core miss my mobile menu links?

axe only evaluates what is rendered. A menu panel that is display none when the page loads is invisible to the check, so its links are never tested. In our reproduction the closed state reported no violations and the open state reported two color-contrast failures. Open every menu, dialog and tab before running axe.

How do I fix many color-contrast violations at once?

Look for a shared design token. In our demo one dim text color produced 34 color-contrast nodes. Changing the token from #646b76 to #7d8590 cleared all of them, because the ratio rose from about 3.4 to between 4.58 and 5.25 on our three dark surfaces, above the 4.5 AA minimum.

What does the scrollable-region-focusable rule mean?

It flags a scrollable container that a keyboard user cannot focus, so they cannot scroll it. The fix is tabindex 0 on the container, plus a role and an accessible label so a screen reader announces it. After adding those attributes our demo page returned zero violations.

Is a clean axe run enough to call a page accessible?

No. axe checks rules a machine can evaluate, such as ratios and missing labels. It cannot judge whether a label is helpful, whether focus order makes sense, or whether a flow works with a screen reader. Treat a clean run as a floor that prevents regressions, and keep manual keyboard and screen-reader testing.

More from Research

• WCAG Contrast Ratio in 20 Lines of Code: Auditing a Dark UI's Color Tokens