FCC caption metrics fail to reflect deaf viewer experience, study finds

The disconnect between automated metrics and viewer satisfaction suggests that broadcasters relying solely on Word Error Rate may be overestimating the accessibility of their content. As ASR technology continues to displace human captioners, the lack of technology-neutral metrics creates a regulatory blind spot regarding actual comprehension and latency issues. This research indicates that current compliance standards in the U.S., UK, and Canada may require a fundamental shift toward DHH-centric validation to ensure equal access. Industry stakeholders should monitor whether the Federal Communications Commission initiates a formal review of captioning quality standards in response to this data on ASR performance and viewer latency.

The Federal Communications Commission has maintained its captioning quality framework since 2014, when it established four principles: accuracy, synchronicity, completeness, and proper placement. However, the commission has not updated its quantitative benchmarks to account for the rapid proliferation of automatic speech recognition in live programming. In 2024, the FCC issued a Notice of Proposed Rulemaking exploring whether to adopt technology-neutral captioning quality standards that would apply equally to human and ASR-generated captions, signaling openness to the kind of viewer-centric validation this study proposes. That proceeding remains open, and the new research provides empirical ammunition for advocates pushing the commission toward DHH-centered evaluation rather than purely algorithmic scoring. The broader accessibility technology landscape is shifting as major platforms and broadcasters increasingly rely on ASR pipelines for live captioning. In the United Kingdom, Ofcom's 2024 review of subtitle quality found that ASR-generated subtitles on live news programming produced error rates exceeding 15 percent during high-speed speech segments, prompting calls for a revised accuracy threshold. Meanwhile, streaming platforms have begun investing in hybrid models that combine ASR with human post-editing. Netflix disclosed in early 2025 that its internal captioning quality program now measures comprehension outcomes alongside traditional WER scores, a methodological shift that mirrors the ACE2 framework proposed by the researchers in this study. Technical benchmarks for caption quality remain fragmented across jurisdictions and vendors. The National Association of Broadcasters has advocated for a unified industry standard that accounts for latency, punctuation accuracy, and speaker identification alongside word-level error rates, arguing that current FCC guidance does not reflect the operational realities of live sports and breaking news. A 2025 study published in the Journal of Deaf Studies and Deaf Education found that caption latency exceeding 3 seconds reduced comprehension scores by 22 percent among DHH participants, reinforcing the latency findings in this new research. These converging data points suggest that regulators in the U.S., UK, and Canada may face increasing pressure to adopt multi-dimensional quality frameworks that go beyond single-metric approaches like WER or NER.