NashTech Blog

Nine New Ways to Fail an Accessibility Audit: WCAG 2.2 for Test Automation Engineers

Table of Contents

WCAG 2.2 became a W3C Recommendation in October 2023. It builds on 2.1 and is backward compatible, so a site that meets 2.2 also meets 2.1 and 2.0. The update adds nine new success criteria and removes one. Regulators and procurement teams increasingly reference 2.2, so teams that only test against 2.1 will start to have gaps.

This post covers what changed, then looks at each new criterion the way a test automation engineer would: what it technically requires, how far it can be automated, and how to test it in code.

What changed in WCAG 2.2

One criterion was removed

4.1.1 Parsing is obsolete and was removed. Modern browsers and assistive technologies handle malformed markup well enough that the criterion no longer protects users.

Nine criteria were added

SCNameLevelWhat it asks for
2.4.11Focus Not Obscured (Minimum)AAA focused element isn’t entirely hidden by sticky headers, cookie banners or chat widgets
2.4.12Focus Not Obscured (Enhanced)AAANo part of the focused element is hidden
2.4.13Focus AppearanceAAAThe focus indicator has a minimum size and contrast
2.5.7Dragging MovementsAAAnything done by dragging also has a single-pointer alternative (click, tap)
2.5.8Target Size (Minimum)AAPointer targets are at least 24×24 CSS pixels, with spacing exceptions
3.2.6Consistent HelpAHelp mechanisms (contact, chat, FAQ) appear in the same relative place across pages
3.3.7Redundant EntryADon’t make users re-enter information they already provided in the same process
3.3.8Accessible Authentication (Minimum)AANo cognitive function tests (remembering or transcribing a password or code) unless an alternative exists
3.3.9Accessible Authentication (Enhanced)AAAStricter version of 3.3.8, with fewer exceptions

The theme is users with low vision, motor impairments, and cognitive or learning disabilities, groups that earlier versions served less well.

What can be automated?

Not everything. Automated tools generally catch only a portion of WCAG issues, and several of the new criteria depend on context. Here is a realistic split:

SCAutomation fit
2.5.8 Target SizeGood. axe-core has a rule, and geometry checks are easy
2.4.11 / 2.4.12 Focus Not ObscuredGood with a custom keyboard-walk test
3.3.8 / 3.3.9 Accessible AuthenticationPartial. You can check paste support, autocomplete attributes and CAPTCHA presence
3.2.6 Consistent HelpPartial. Compare help-link position across pages
3.3.7 Redundant EntryPartial. Scripted per flow
2.5.7 Dragging MovementsPartial. You can verify an alternative exists and works, but not that one is needed
2.4.13 Focus AppearancePartial. Check that styles exist, but area and contrast need review

Treat automated tests as a regression safety net, and keep manual review for the rest.

WCAG 2.2 Criteria Through a Test Automation Lens

Each criterion below is covered in four parts. Technical Essence states what the criterion measures and its exceptions. Typical Failure gives a concrete example of what breaking it looks like. Automation Perspective & Challenge says what a script can and cannot decide. Test Approach describes what to automate, with a small snippet only where it helps, and what still needs a person. The examples assume Playwright with TypeScript and @axe-core/playwright, and a Page Object Model: page objects own locators and DOM probing, and specs own the assertions.

1. SC 2.4.11 Focus Not Obscured (Minimum), Level AA

Technical Essence

Only author-created content counts as an obstruction (not browser UI), and partial overlap is allowed at this level. The usual cause is a fixed element covering content that the browser scrolled into view, and the usual fix is scroll-padding or scroll-margin.

Typical Failure

Tabbing to a “Submit” button scrolls it under a sticky header or cookie banner, so a keyboard user can no longer see where focus is.

Automation Perspective & Challenge

Well suited to automation, because the outcome is geometric and repeatable. The challenge is coverage: the overlay must be present during the test (cookie banner open, chat widget expanded), and the result depends on viewport size, so it should run at desktop and mobile sizes. Content the user can dismiss or move is an exception, and a script cannot judge whether dismissal really works without a separate interaction test.

Test Approach

Automate: press Tab repeatedly, and Shift+Tab for the reverse pass. After each press, sample points across the focused element’s bounding box and ask the browser what is on top at each point. If none of the points hit the element or one of its descendants, it is fully obscured. Put the walk in a BasePage method that returns data, and let the spec assert that no entry has a visible ratio of zero, at desktop and mobile viewports and with the cookie banner and chat widget open.

const hit = document.elementFromPoint(x, y);
const visible = !!hit && focused.contains(hit);

Verify manually: confirm that any obscuring element the user can dismiss or move really can be dismissed or moved.

2. SC 2.4.12 Focus Not Obscured (Enhanced), Level AAA

Technical Essence

Same scope as 2.4.11 (keyboard focus, author-created content), but the threshold moves from “not entirely hidden” to “not hidden at all”, so even a one-pixel overlap fails.

Typical Failure

A focused link sits half behind a fixed chat widget. It passes at AA because it is not entirely hidden, but fails here.

Automation Perspective & Challenge

The same keyboard walk works, with a stricter threshold. Because sampling is discrete, a thin overlap can fall between sample points, so a denser grid reduces false passes but never removes them. Expect more failures than at AA, so keep it out of the pull-request gate.

Test Approach

Automate: reuse the 2.4.11 walk with a denser grid (for example 5×5) and require every sample to be visible. Tag the spec @aaa so CI can run it separately with npx playwright test --grep @aaa.

Verify manually: do a keyboard walkthrough at 200% zoom to catch thin overlaps that sampling misses.

3. SC 2.4.13 Focus Appearance, Level AAA

Technical Essence

Two measurable conditions. Area: the indicator must be at least as large as a 2 CSS pixel perimeter around the unfocused component. Contrast: the pixels that change between the focused and unfocused states must have at least 3:1 contrast. Unmodified browser default indicators are exempt.

Typical Failure

A custom button removes the default outline with outline: none and replaces it with a faint background tint, or a 1px light-gray outline on a white page.

Automation Perspective & Challenge

This is the hardest focus criterion to automate. Computed styles tell you whether an outline exists and how thick it is, but not the indicator’s area or the contrast change between states. Indicators built from box-shadow, borders, background changes or pseudo-elements are invisible to an outline check, and a ring on an image or gradient has no single background color to measure against.

Test Approach

Automate: treat the test as a guard against missing or thin indicators, not as proof of compliance. Tab through the page, read outline-style and outline-width from getComputedStyle, and fail when the outline is absent or under 2px. When the element relies on box-shadow instead, do not fail; add a manual-review annotation.

Verify manually: judge the indicator’s real area and contrast by eye, or compare screenshots of the focused and unfocused element.

4. SC 2.5.7 Dragging Movements, Level AA

Technical Essence

A dragging movement is any press, move, release sequence, so it covers sortable lists, sliders and resizable panels, not just drag-and-drop. The alternative must work with a single pointer and no dragging (click, tap, menu), and it can be a different control on the same page, such as a text input beside a slider. A keyboard-only alternative or a swipe gesture alone does not satisfy it. Exceptions: the drag is essential (freehand drawing, for example), or the behavior is determined by the user agent and not modified by the author, such as the browser’s own scrolling.

Typical Failure

A Kanban board where cards can only be moved between columns by dragging, with no “Move to” menu or buttons.

Automation Perspective & Challenge

A script can find drag interactions and verify that an alternative works. It cannot decide whether a drag is essential, and it cannot find drag handlers that are not exposed in the DOM, such as custom pointer-event code on a canvas. Libraries differ in how they mark draggable elements.

Test Approach

Automate: do two things. First, build an inventory: count elements matching [draggable="true"] and your library’s attributes (react-beautiful-dnd, dnd-kit) on the pages in scope, and report them as annotations so new drag UI is noticed. Second, write a functional test per drag feature that uses only the alternative: open the card’s “Move to” menu, pick a column, and assert the card appears there. Add a separate keyboard check for the same feature: keyboard access is its own requirement (2.1.1) and does not satisfy this criterion on its own, which needs a click or tap path.

Verify manually: decide whether each drag is essential, and look for custom drag code the inventory can’t see.

5. SC 2.5.8 Target Size (Minimum), Level AA

Technical Essence

A target meets the size if a 24×24 CSS pixel square fits entirely inside it, and zooming does not change that. A smaller target still passes if a 24px circle centered on its bounding box does not intersect another target or another undersized target’s circle (the spacing exception), if an equivalent control on the same page meets the size, if it sits in a sentence or its size is constrained by the line-height of surrounding text (the inline exception), if its size is set by the user agent and not modified by the author, or if the size is essential or legally required. A slider or color picker counts as one target.

Typical Failure

An 18×18px icon-only “Add card” or “Close” button sitting right next to other small controls, so neither the size nor the spacing exception applies.

Automation Perspective & Challenge

Strong candidate. axe-core ships a target-size rule that understands the spacing exception, and raw geometry is easy to measure (a bounding-box check only approximates the rule that a 24px square must fit inside the target, so rounded or skewed shapes can differ). The challenge is exceptions: custom geometry checks over-report because they ignore spacing, unstyled native controls and equivalent controls elsewhere on the page. Older axe-core versions lack the rule, and skipping it silently makes a green run meaningless.

Test Approach

Automate: run the axe rule on each key page at desktop and mobile viewports (set as Playwright projects so specs do not loop over sizes). Add a custom geometry check that lists visible buttons, inputs, selects and non-inline links under 24×24px, then compare it with axe’s findings. Use soft assertions so both results are reported.

const { violations } = await new AxeBuilder({ page }).withRules(['target-size']).analyze();
expect(violations).toEqual([]);

Verify manually: check geometry-only findings against the exceptions (native unstyled controls, equivalent controls, essential presentation), and confirm your axe-core version includes the rule.

6. SC 3.2.6 Consistent Help, Level A

Technical Essence

The requirement is about relative order to other page content, not pixel position, and it only applies to pages in the same set. Help mechanisms are human contact details, a human contact mechanism, a self-help option such as an FAQ, or an automated contact mechanism such as a chatbot. The user can still change the position, and the position can legitimately differ at different breakpoints.

Typical Failure

The “Contact us” link is in the header on the home page but in the footer on checkout pages, or the chat launcher is bottom-right on some pages and bottom-left on others.

Automation Perspective & Challenge

Partial. Comparing positions across pages is simple once you have identified the help element. The difficulty is identification and scope: “the help link” is a text or label heuristic, and deciding which pages belong to the same set is a product decision, not a technical one. Position also changes with viewport, so comparisons must stay within one breakpoint.

Test Approach

Automate: create a HelpWidget component object that locates the help link and the chat launcher. For each page in the set, record the link’s landmark (header, footer or nav) and its index among that landmark’s links, and the launcher’s offset from the viewport corner. In the spec, assert that landmark and index are identical across pages and that the launcher stays within a small tolerance. Repeat at mobile and tablet viewports.

Verify manually: decide which pages belong to the same set, and whether the help mechanisms found are the ones users rely on.

7. SC 3.3.7 Redundant Entry, Level A

Technical Essence

It covers information the user entered or was shown earlier in the same process, such as a multi-step checkout or sign-up. Either auto-populating the field or offering the earlier value as a selectable choice satisfies it. Exceptions: re-entry is essential, required for security (confirming a new password), or the earlier value is no longer valid.

Typical Failure

Step 2 of checkout asks for the billing address again with no “Same as shipping” option, or going back to step 1 clears what the user typed.

Automation Perspective & Challenge

Partial. Each multi-step flow needs its own script, because there is no generic way to know which fields repeat. The decision about whether a repeated field falls under an exception is human. A script can, however, check the mechanics: pre-filled values, a “Same as shipping” control, and values that survive going back a step.

Test Approach

Automate: model the flow in a page object, for example a CheckoutPage with fillAddress, next, back, addressValues and reuseShippingIfOffered. One test fills shipping, continues, applies the reuse control if present, and asserts the billing values equal the shipping data. Another goes back a step and asserts the earlier values are still there. Use expect.poll on the values rather than fixed waits. As a static complement, scan the repository’s multi-step forms for fields with the same name or autocomplete in more than one step.

Verify manually: judge whether each remaining repeated field is covered by an exception (essential, security, or no longer valid).

8. SC 3.3.8 Accessible Authentication (Minimum), Level AA

Technical Essence

A cognitive function test, such as remembering a password or solving a puzzle, is allowed in an authentication step only if that step also provides an alternative method without such a test, or a mechanism that assists the user. Two further exceptions apply at this level: object recognition (for example “select all traffic lights”) and identifying non-text content the user supplied to the site. Every step counts, including multi-factor steps and account recovery, but creating an account is out of scope. Password managers and copy and paste are the classic assistive mechanisms, so blocking them fails the criterion unless an alternative exists.

Typical Failure

A login form that blocks pasting into the password field, or shows a distorted-text CAPTCHA with no alternative. An audio CAPTCHA that still has to be transcribed doesn’t count as one.

Automation Perspective & Challenge

Partial. The enabling mechanics are testable: correct autocomplete values, type="password", paste not blocked, and an OTP field that allows autofill. Whether a CAPTCHA or a “third character of your PIN” prompt is acceptable is a judgment about the exceptions, so a script can only detect and flag them.

Test Approach

Automate: put the locators and helpers in AuthPage, LoginPage and MfaPage objects. Assert that the username field has autocomplete of username or email, the password field has current-password, and the OTP field has one-time-code. Check paste two ways: dispatch a synthetic paste event and inspect defaultPrevented, then do a real clipboard paste and assert the value arrives. Detect CAPTCHA widgets and report them as manual-review annotations rather than failures.

const prevented = await password.evaluate((el) => {
  const e = new ClipboardEvent('paste', { clipboardData: new DataTransfer(), bubbles: true, cancelable: true });
  el.dispatchEvent(e);
  return e.defaultPrevented;
});
expect(prevented).toBe(false);

Verify manually: decide whether each flagged challenge (CAPTCHA, PIN-character prompt) qualifies under the object-recognition or personal-content exceptions, and whether an alternative exists, such as a passkey or an emailed link. An audio CAPTCHA that still requires transcription is not an alternative.

9. SC 3.3.9 Accessible Authentication (Enhanced), Level AAA

Technical Essence

Identical to 3.3.8 (including its scope, which covers authentication of existing users and not account creation) except for the two exceptions it drops, object recognition and personal content. Every cognitive challenge now needs an alternative or an assistive mechanism.

Typical Failure

A login page with an image-selection CAPTCHA and no passkey or magic-link option, or a password-recovery page that asks a security question. Both can pass at AA under the object-recognition and personal-content exceptions but fail here.

Automation Perspective & Challenge

Because the exceptions are gone, detection becomes a reasonable failure condition rather than a flag: a CAPTCHA, image-selection challenge or security question on the page is a finding. The remaining limits are detection coverage (custom challenge widgets look like ordinary markup) and recognizing what counts as a non-cognitive route.

Test Approach

Automate: reuse AuthPage. On the login, MFA and password-recovery pages, assert that there are zero CAPTCHA elements and no security-question text. Then assert that a non-cognitive route exists, such as a visible passkey, magic-link or SSO button. Tag the specs @aaa.

Verify manually: look for custom challenge widgets the selectors miss. Report a real failure, such as an image CAPTCHA on the login page, as an accessibility finding with alternatives, not something to work around in the test.

Wiring it into CI

Run the AA specs on every pull request, upload the Playwright HTML report as an artifact, and fail the job on any test failure. Run the @aaa specs in a separate scheduled job, weekly for example, so a failing enhanced check does not block pull requests that only target AA. Pair it with a PR template that asks reviewers to confirm the manual checks for criteria that automation can’t settle.

Guardrails that keep these tests honest

  1. Break something on purpose. Remove an autocomplete attribute or shrink a button, and confirm the test fails. A test that has never failed may be passing trivially.
  2. Separate test fixes from app fixes. When a test fails, decide whether the test is wrong (flaky locator, missing wait) or the page is. Real accessibility bugs should be reported, not worked around in the test.
  3. Require test.fixme for gaps. A visible skipped test is better than a falsely green one.
  4. Check exceptions against the W3C “Understanding” documents, not memory.
  5. Keep humans in the loop. Screen reader testing and keyboard walkthroughs remain essential. Automation widens coverage, but it doesn’t replace judgment.

Wrapping up

WCAG 2.2 is a modest set of changes, but it targets real barriers: obscured focus, tiny tap targets, drag-only interactions, memory-heavy logins and repetitive forms. Looked at through a test automation lens, some are strong regression candidates (target size, focus not obscured), some are partial (authentication, redundant entry, consistent help), and some mostly need human review (focus appearance, whether a drag is essential). Automate what is deterministic, flag what isn’t, and keep a clear list of what still needs a person.

Picture of Ty Ngo Hoang

Ty Ngo Hoang

Automate and Chill!

Suggested Article

Scroll to Top