Skip to content
What an Accessibility Conformance Score Cannot Tell You About Customer Experience
Most accessibility investment follows one path. An organisation commissions an audit, works through the findings, ships the fixes, and reaches the middle conformance level (AA) of the Web Content Accessibility Guidelines (WCAG). Months of work sit behind that. Real money too. At the end the organisation holds the exact thing procurement, regulators and boards all ask for: WCAG AA compliance.
However, the customer numbers stay where they were.
Contact volumes hold steady, abandoned baskets hold steady, and nobody in the business can point at a screen and say disabled customers now get further than they used to. The explanation sits inside the standard. Those guidelines were built to settle a question about access: can disabled people reach and operate what everybody else reaches and operates? Your customers are answering a different one: did they finish what they came to do?
Can accessibility be measured as a percentage?
Every organisation funding this work eventually asks for a score. A percentage travels well. It fits a dashboard, compares across quarters, and survives a trip to the board.
The United States Department of Justice considered a fixed percentage score: plain counts of failures, and weighted variants scoring them by severity or by how heavily the content gets used. But then, they declined all of them: "A single instance of nonconformance might prevent a person with a disability from accessing a particular service, program, or activity."
There sits the arithmetic problem with any accessibility score. The denominator swallows what matters. A large site can carry a handful of barriers that lock people out of the only task they came for, and still post a number in the high nineties.
Passing 92% of your tests is a legitimate statement about a measurement. Being 92% compliant is a statement saying wholly nothing. None of which argues against measuring, and organisations should track far more than they currently do: defect counts, severity, where defects enter, how long they stay open. Better yet, track some people metrics: percentage of employees who have completed accessibility training, number of accessibility champions successfully onboarded.
Does WCAG AA Compliance Mean Your Website Is Accessible?
Compliance and Conformance Are Two Different Words
The standard only ever claims the second. WCAG defines conformance and sets three levels of it. Compliance is what a law, a contract or a procurement form demands, and it borrows the standard to say what it means. An organisation can therefore satisfy every rule it is held to while a customer still cannot check out.
Where the Success Criteria Stop
This next point gets misread constantly, usually as an attack on the standard. It sits closer to a plain reading of what the standard was built to do. Parity of experience is the design intent. Experience quality was not in scope.
Work through the success criteria and that intent becomes plain. Success Criterion 1.4.1 applies only where colour carries information, so a link styled identically to the text around it passes, though nobody can tell it is a link. Compliant alt text is about detecting presence not effectiveness. Usable alt text is meaningful, descriptive and useful. Readability sits at Success Criteria 3.1.3 to 3.1.6, all of them at the highest level (AAA), which leaves content at the middle level free to be as dense and acronym-stuffed as its author likes.
So there is a more honest way to describe what conformance delivers. A measurable baseline.
How Much of What Disabled Users Struggle With Does WCAG Cover?
One number anchors this argument, and it needs careful handling. Task-based evaluations run at the University of York in 2012 put 32 blind participants across 16 websites and logged 1,383 separate user problems. Success Criteria in WCAG 2.0 covered 50.4% of them. Where the standard did cover a problem and the site had implemented the recommended technique, the technique often failed to fix what the person was struggling with.
Has WCAG 2.2 Closed the Gap?
Since then the standard has moved twice. WCAG 2.1 added 17 success criteria in 2018, aimed at mobile, low vision, and cognitive and learning disabilities. WCAG 2.2 added nine more in 2023 and retired 4.1.1 Parsing. Reflow, Text Spacing, Target Size, Accessible Authentication: real barriers addressed by successive versions of the standard. Any article claiming nothing has improved since 2012 deserves ignoring.
Look at which barriers closed, though. Almost every one of those 26 additions sits under Perceivable or Operable. Readability has not moved from the highest level since WCAG 2.0. And the six uncovered problem types from that study, content turning up where users did not expect it, content missing where they did, pages too slow, inaccessible or unnecessarily restrictive PDFs, information architecture too complex to navigate, and broken links, all sit outside WCAG 2.2 today. The percentage has aged. The shape of the gap has not.
The Experiment That Cuts the Other Way
One study cuts hard against this and belongs here. A 2017 experiment tested 110 participants, half with visual impairments, across three conformance levels, and found higher conformance produced better task completion and warmer ratings from both groups. The nearest thing to a replication followed in 2023, putting 131 participants on an online shop built at non-conformant and AA versions, and reported no statistically significant effect on usability or user experience.
Both findings stand. The York tradition watches people use real sites, then asks which of their problems a standard would have caught. The Swiss experiments build one site, move its conformance level, and measure what changes. Neither result says conformance does harm. Conformance helps. Finishing the job takes something else.
What Do Disabled Customers Report About Sites That Meet the Standard?
WebAIM's tenth screen reader user survey gathered 1,539 valid responses over the turn of 2024. Asked what would most improve their experience, 85.9% chose better built websites, far ahead of any improvement to the software they use.
Fable adapted the System Usability Scale for assistive technology users and scored 2,100 sessions with it during 2022. Screen reader users averaged 56 against a general population benchmark of 68. Half of them scored 56 or below. Among the general population, a fifth did.
Journey testing shows the mechanism at work. Equal Entry and Substrakt tested ticket buying and donation on a New York arts venue's site in 2026, and watched users tab between 30 and 50 times to reach a date, because focus never moved to the first available one. Every component on that page could pass its criteria alone. The journey still failed.
What to Measure Instead of a Conformance Score
Measurement is not the problem here. Nothing improves on its own, and an organisation that measures nothing cannot show progress, argue for budget, or notice a regression before a customer does. A team that cannot say how accessible its product is has usually answered the question already.
The single number is what needs replacing. Swap the component question for the task question. "Does this form have labels" is answerable by a scanner and tells you almost nothing. "Can someone with no vision complete and submit this form" is answerable only by watching somebody try.
→ Test with disabled people, and make it the primary instrument. Recruit participants who use their own assistive technology every day, and watch them attempt the journeys carrying your revenue. Colleagues simulating that software are no substitute.
→ Measure task completion rather than defect counts. Who finished, how long they took, where they gave up.
→ Report the gap. Completion for assistive technology users against completion for everybody else, same journey, same week. A gap is a business number.
→ Track people and process alongside product. These are initiatives like training and accessibility champions, how early accessibility is entering at the design stage, and then the product measures like the cadence of accessibility testing and their outcomes, or accessibility-related complaints.
The Question Worth Taking to Your Next Review
Conformance and completion answer different questions, and most organisations have bought only the first. Buying one alone leaves you holding a defensible legal position and no evidence at all about your customers.
So: what is your task completion rate for screen reader users on your highest value journey, and by how many percentage points does it sit below everybody else's?
Next Step
REO Digital runs accessibility research with disabled participants on live customer journeys, and reports completion for those participants against completion for everybody else on the same journey. See how REO Digital approaches digital accessibility.
Related Reading
FAQs
What is WCAG AA compliance?
WCAG AA compliance means a website meets every Level A and Level AA success criterion in the Web Content Accessibility Guidelines, the W3C standard for web accessibility. AA is the middle of three conformance levels and is the level most laws, contracts and procurement processes reference. The guidelines themselves use the word conformance rather than compliance: conformance describes meeting the standard, and compliance describes meeting whatever legal or contractual rule points at it.
Does WCAG AA conformance mean a website is accessible?
WCAG AA conformance means a website meets a defined set of testable success criteria at that level. Conformance does not establish that disabled people can complete tasks on the site. The standard secures access on equal terms. It does not judge the quality of what is accessed, so a site can conform while remaining slow, densely written or hard to navigate for everyone. A more honest description of what conformance delivers is a measurable baseline.
Is WCAG compliance a legal requirement in the UK?
UK public sector bodies must meet WCAG 2.2 at level AA under the Public Sector Bodies (Websites and Mobile Applications) (No. 2) Accessibility Regulations 2018, according to GOV.UK guidance checked on 17 September 2026. Private sector organisations are not held to a named WCAG version by statute, but the Equality Act 2010 requires reasonable adjustments for disabled people, and WCAG is the standard most commonly used as evidence of them. REO Digital is not a law firm, and organisations should take their own legal advice on what applies to them.
What is the difference between accessibility and usability?
Accessibility determines whether a person can reach and operate a product at all. Usability determines how well they achieve what they came to do. Assessing conformance means checking a product against a standard such as WCAG. Assessing usability means watching people attempt real tasks. A 2007 study comparing blind and sighted participants on the same websites found their problems formed two intersecting sets with roughly 15% overlap.
Why has our accessibility investment not improved customer experience metrics?
Accessibility conformance work measures whether a product meets success criteria, which is a different measurement from whether disabled customers complete journeys. An audit reports defects against a standard, while customer experience reporting covers task completion, contact volume and conversion. An organisation funding conformance alone has bought a parity instrument and will see parity results. Measuring task completion with assistive technology users on revenue-carrying journeys produces figures customer experience reporting can use.
How do you test accessibility with disabled users?
Accessibility user testing recruits disabled participants who use their own assistive technology daily, gives them real tasks on real journeys, and records whether they finished. That differs from an accessibility audit, which checks a product against success criteria without anyone attempting a task. Run it on the journeys that carry revenue, report task completion for assistive technology users against completion for everybody else, and repeat it after each release. Colleagues simulating assistive technology produce a poor substitute, because fluency with the software changes what a person can do with it.
Sources
Karl Groves, "Accessible is conformant", AFixt, 12 August 2026. https://afixt.com/accessible-is-conformant/
United States Department of Justice, "Nondiscrimination on the Basis of Disability: Accessibility of Web Information and Services of State and Local Government Entities", final rule, 24 April 2024. https://www.federalregister.gov/documents/2024/04/24/2024-07758/nondiscrimination-on-the-basis-of-disability-accessibility-of-web-information-and-services-of-state
Craig Abbott, "5 reasons why WCAG AA compliance does not mean your website is accessible", 27 November 2025. https://www.craigabbott.co.uk/blog/2025/5-reasons-why-wcag-aa-compliance-does-not-mean-your-website-is-accessible/
Christopher Power, André Freire, Helen Petrie and David Swallow, "Guidelines are only half of the story: accessibility problems encountered by blind users on the web", CHI 2012, University of York. https://pure.york.ac.uk/portal/en/publications/guidelines-are-only-half-of-the-story-accessibility-problems-enco/
Sven Schmutz, Andreas Sonderegger and Juergen Sauer, "Implementing Recommendations From Web Accessibility Guidelines: A Comparative Study of Nondisabled Users and Users With Visual Impairments", Human Factors 59(6), 2017. https://journals.sagepub.com/doi/abs/10.1177/0018720817708397
Beat Vollenwyder and colleagues, "How compliance with web accessibility standards shapes the experiences of users with and without disabilities", International Journal of Human-Computer Studies 170, 2023, open access. https://edoc.unibas.ch/entities/publication/bb13f912-f52c-40a3-b7fe-b0c170a2c4bb
Helen Petrie and Omar Kheir, "The relationship between accessibility and usability of websites", CHI 2007, University of York. https://pure.york.ac.uk/portal/en/publications/the-relationship-between-accessibility-and-usability-of-websites-2/
W3C Web Accessibility Initiative, "What's New in WCAG 2.1". https://www.w3.org/WAI/standards-guidelines/wcag/new-in-21/
W3C Web Accessibility Initiative, "What's New in WCAG 2.2". https://www.w3.org/WAI/standards-guidelines/wcag/new-in-22/
WebAIM, "Screen Reader User Survey #10 Results", fieldwork December 2023 and January 2024. https://webaim.org/projects/screenreadersurvey10/
Fable, "Usability testing: benchmarking accessibility, a year in review", 2022 sessions. Vendor research, drawn from Fable's own panel of assistive technology users. https://makeitfable.com/article/usability-testing-benchmarking-accessibility-a-year-in-review/
Equal Entry with Substrakt, "Accessibility user journey testing", 4 August 2026. Agency case study. https://equalentry.com/accessibility-user-journey-testing-pac-nyc/
Lucy Ruck, "Do digital accessibility metrics matter, and what happens when we don't have them?", Business Disability Forum, 21 May 2026. Membership body guidance, source of the people, process and product framing. https://businessdisabilityforum.org.uk/do-digital-accessibility-metrics-matter-and-what-happens-when-we-dont-have-them/
"Don't fall into the accessibility grade/score trap", Access * Ability newsletter, 10 December 2025. Author not named on the page. https://buttondown.com/access-ability/archive/dont-fall-into-the-accessibility-gradescore-trap/
GOV.UK, "Understanding accessibility requirements for public sector bodies", checked 17 September 2026. https://www.gov.uk/guidance/accessibility-requirements-for-public-sector-websites-and-apps
