Automate Accessibility Testing with Pa11y

Facing Accessibility Issues? Automate Audits with Pa11y

You spent weeks on layout, but a client using a screen reader can't see the 'Buy' button. Or worse—received a complaint from a regulatory authority for WCAG 2.1 non-compliance. Manual checking of 500 pages means days of QA work. Pa11y solves this: it runs in CLI, reads sitemap.xml, and produces a report for all pages in one pass. We've been using Pa11y in CI/CD for over 5 years across 50+ projects—sharing our experience. Automating website accessibility testing with Pa11y reduces audit time by 70%, and maintenance costs drop due to early bug detection. See the savings: on a 2000-page project, we cut audit from 3 days to 2 hours and found 47 critical errors that QA missed, saving an estimated $5,000 in QA time.

Why Automate Accessibility Testing?

Without automation, you'll miss 30–50% of violations. Pa11y checks color contrast, alt texts, ARIA attributes, keyboard navigation. We configure it to never miss critical errors while ignoring false positives (e.g., contrast on disabled elements). Typical issues: missing image alt, incorrect ARIA roles, low text contrast. Pa11y automated accessibility testing for WCAG 2.1 compliance reduces audit costs and speeds up releases.

How to Integrate Pa11y into CI/CD?

Setup takes 1–2 days and includes five steps:

1. Analysis—gather URLs from sitemap.xml, define the standard (usually WCAG 2.1 AA).
2. Configuration—create .pa11yci.json with required parameters and exclusions.
3. Integration—add pa11y-ci command to your pipeline (GitLab CI, GitHub Actions, Jenkins).
4. Testing—run a trial audit, adjust false positives.
5. Documentation—document the workflow for the team.

Analysis and Configuration

The process starts with analyzing your site: collect URLs from sitemap.xml, etc. Define the standard (usually WCAG2AA), timeouts, exclusions. Output: .pa11yci.json:

// .pa11yci.json
{
  "defaults": {
    "standard":  "WCAG2AA",
    "timeout":   30000,
    "wait":      1000,
    "ignore":    [
      "WCAG2AA.Principle1.Guideline1_4.1_4_3.G18.Fail"
    ],
    "chromeLaunchConfig": {
      "args": ["--no-sandbox", "--disable-setuid-sandbox"]
    }
  },
  "urls": [
    "https://example.com",
    "https://example.com/about",
    "https://example.com/contact",
    {
      "url":    "https://example.com/login",
      "actions": [
        "wait for element #login-form to be visible"
      ]
    }
  ]
}

Pipeline Integration

Add a step in CI: on GitLab—pa11y-ci --config .pa11yci.json --threshold 5, on GitHub Actions—similarly. The --threshold parameter sets the allowed number of errors. Strict mode (--threshold 0) means the pipeline fails on any error.

Reading from sitemap.xml

pa11y-ci --sitemap https://example.com/sitemap.xml \
  --sitemap-find "https://example.com" \
  --sitemap-replace "http://localhost:3000" \
  --threshold 0

How to Ignore False Positives?

Pa11y sometimes complains about contrast on disabled elements or ARIA roles set by the framework. We add an ignore list in the config—tailored to your UI kit. For example:

"ignore": [
  "WCAG2AA.Principle1.Guideline1_4.1_4_3.G18.Fail",
  "WCAG2AA.Principle4.Guideline4_1.4_1_2.H91.InputSearch.Name"
]

This eliminates up to 90% of false positives without losing critical checks.

Comparison: Pa11y vs axe-core

Pa11y wins when you need to scan the entire site. Axe is preferable in unit tests. We combine them: Pa11y for nightly audits, axe in pre-commit hooks. This gives full coverage without duplication. Compared to manual testing, Pa11y is 3 times faster and reduces cost by 70%.

// scripts/a11y-audit.js
const pa11y   = require('pa11y');
const fs      = require('fs');

const PAGES = [
  { url: 'http://localhost:3000', name: 'Home' },
  { url: 'http://localhost:3000/catalog', name: 'Catalog' },
  { url: 'http://localhost:3000/checkout', name: 'Checkout' },
];

async function audit() {
  const results = [];

  for (const page of PAGES) {
    console.log(`Checking: ${page.name}`);
    const result = await pa11y(page.url, {
      standard:  'WCAG2AA',
      timeout:   20000,
      actions:   page.actions || [],
    });

    results.push({
      name:       page.name,
      url:        page.url,
      issues:     result.issues.length,
      critical:   result.issues.filter(i => i.type === 'error').length,
      warnings:   result.issues.filter(i => i.type === 'warning').length,
      violations: result.issues,
    });
  }

  fs.writeFileSync('a11y-report.json', JSON.stringify(results, null, 2));

  console.table(results.map(r => ({
    Page: r.name,
    Errors: r.critical,
    Warnings: r.warnings,
  })));

  if (results.some(r => r.critical > 0)) {
    process.exit(1);
  }
}

audit();

What You Get

After Pa11y setup, you receive:

• A working .pa11yci.json configuration file tailored to your project.
• CI/CD integration—automatic audit on every push.
• Custom ignore list for false positives.
• Documentation on interpreting reports and fixing common errors.
• Team training: how to read Pa11y reports and fix accessibility bugs.

Pa11y Setup Stages

Timelines and Cost

Basic setup takes 1–2 days. Extended setup with custom rules, screenshots, and training—up to 5 days. Cost is calculated individually based on site size; typical investment is $2,000–$5,000. Payback period is less than 3 months. We'll assess your project in 1 hour—contact us.

As per WCAG 2.1 guidelines, Pa11y launches a headless browser (Chrome), loads each page, and checks it against the selected WCAG standard. Results are grouped by error type: error (critical), warning (advisory), notice (informational). This allows quick identification of problem areas and prioritization of fixes.

How to Start?

Contact us for a consultation. Order an accessibility audit, and if you receive a fine for WCAG non-compliance after our audit, we'll recheck for free. This guarantee ensures peace of mind. Our team has over 5 years of experience and has helped 50+ companies achieve compliance. Investment in automation pays off within 2-3 months. Get your Pa11y configuration and eliminate manual checks forever.

Website Accessibility: WCAG, Screen Readers, Keyboard Navigation

On a major bank's website, the "Submit Application" button was marked up as <div class="btn" onclick="...">. The NVDA screen reader did not announce it, Tab skipped it, Enter didn't work. For thousands of blind users, this bank simply did not exist as an online service. We see such problems every day in dozens of projects — and developing accessible websites according to WCAG 2.2 AA has become the only way to avoid discrimination and legal risks. Fines for non-accessibility for legal entities can reach substantial amounts, and lawsuits millions.

In this card — how we make web accessibility a11y work, based on real cases, with a specific tech stack and numbers. No generic phrases.

Why is Semantic Markup the Foundation of Web Accessibility (a11y)?

Most accessibility problems are solved by correct HTML, not additional ARIA attributes. <button> instead of <div onclick>, <nav> instead of <div class="navigation">, <h1>–<h6> in proper hierarchy, <label for="field-id"> instead of <div class="label">. This is the basic level, but in practice, every second form in Russian online stores does not have correct <label> tags.

ARIA is needed where native HTML falls short: custom components — dropdown menus, tooltips, modal windows, tabs, accordions. And here the complexity begins.

A typical mistake in custom dropdowns: the screen reader does not know it is a combobox, does not announce the number of options, does not say which one is selected, focus does not move to the list when opened. Proper implementation:

• role="combobox" on the input
• aria-expanded="true/false" when opened/closed
• aria-controls="listbox-id" points to the list
• aria-activedescendant — ID of the currently selected item
• role="option" and aria-selected on each option

This is not theory; it is tested with a screen reader. NVDA + Chrome or VoiceOver + Safari is a mandatory part of QA.

<div role="combobox" aria-expanded="false" aria-controls="listbox-1" aria-activedescendant="" tabindex="0">
  <label for="input-1">Select city</label>
  <input id="input-1" type="text" role="combobox" aria-autocomplete="list" />
  <ul id="listbox-1" role="listbox" aria-label="Cities">
    <li role="option" aria-selected="false" id="opt-1">Moscow</li>
    <li role="option" aria-selected="false" id="opt-2">St. Petersburg</li>
  </ul>
</div>

The cost of fixing a single Level A violation varies depending on complexity. Implementing a11y from the design stage reduces the refactoring budget by 2–3 times compared to retrofitting a finished site.

How to Properly Build Keyboard Navigation?

Tab order should match the visual order of elements. If in HTML the "Cancel" button comes before "Confirm", but CSS swaps them — the keyboard user is confused.

Focus trap in modal windows. When a modal opens, Tab should cycle only within it. When closing, return focus to the element that opened the modal. Without this, the user ends up at the top of the page after closing.

tabindex="-1" — element does not enter Tab sequence but can receive focus programmatically. Used for elements that receive focus via JavaScript (section headings after anchor navigation).

tabindex="1" and above is almost always an error. Explicit order breaks natural order and creates unpredictable behavior. Control order via DOM, not tabindex.

Skip links — a "Skip to content" link, hidden visually, visible on Tab. Allows screen reader users to skip repetitive navigation.

Color and Contrast: Requirements and Common Violations

WCAG 2.2 AA requires contrast 4.5:1 for normal text, 3:1 for large text (18px+ or 14px+ bold). AAA requires 7:1 and 4.5:1.

The most common violations: gray placeholder in inputs (#999 on white = 2.9:1), light gray secondary text, white text on pastel backgrounds.

Color should not be the sole indicator: "required fields are red" without an asterisk — violation for color blind users.

Testing tools: axe DevTools, WAVE, Accessibility Inspector in Chrome DevTools. axe-core integrates into Playwright tests: automatic check of 80+ rules on every deployment. Manual testing finds about 60% more errors than automated.

What Is Important About Media Content and Dynamics?

Images without alt — a common basic failure. alt should be meaningful: not alt="image_123.jpg", but a description of content relevant to context. Decorative images — alt="" (empty, not missing attribute).

Video should have captions. YouTube auto-captions are not a standard; they make mistakes. WebVTT files with correct captions for all educational and marketing video content.

Animations — a problem for users with vestibular disorders. @media (prefers-reduced-motion: reduce) — media query that disables or slows animations for users with that OS setting.

What Changed in WCAG 2.2?

Version 2.2 came into effect with new criteria:

These criteria raise the entry bar, but we already include them in our standard checklist.

Audit and Remediation

Automated tools find about 30–40% of violations. The rest is only manual testing. Minimum scenario: go through the entire critical user flow (registration, purchase, form) using only keyboard and screen reader.

Process

1. Automated audit — axe-core, Lighthouse, WAVE — outputs 80+ rules.
2. Manual testing — NVDA, VoiceOver, keyboard — 2–3 days for a typical site.
3. Violation prioritization — P1 (blocks usage), P2 (creates difficulties), P3 (enhancements).
4. Fixing — iteratively, integrate checks into CI via Playwright + axe.
5. Re-audit — close all P1/P2 before release.
6. Documentation and handover — report with results, maintenance recommendations, team training.

Results and Scope

• Full audit report with violation prioritization (PDF/HTML)
• Fixed code: semantic markup, ARIA, keyboard navigation
• Integration of axe-core into CI/CD for regression control
• Training for client developers on a11y (2-hour session)
• Access to repository with correct component examples
• Guarantee of WCAG 2.2 AA compliance at time of delivery

Timeline

Budget is calculated individually after the audit. Contact us — we will evaluate your project in 1 day. Get a consultation and free checklist when ordering an audit.

Experience and Guarantees

We have been working in web accessibility a11y for over 8 years. Completed more than 50 projects for banks, retail, and government. Certified specialists (IAAP CPACC, WAS). We guarantee passing a third-party audit or will fix for free.

WCAG 2.2 Standard — official W3C recommendation defining web content accessibility requirements.

Wikipedia: Web Content Accessibility Guidelines
Wikipedia: ARIA

— web accessibility levels a11y per version 2.2.

Order an audit now — get a checklist and preliminary estimate for free. Contact us – we will respond within an hour.