Position: Assistive Agents Need Accessibility Align... | AI Research

Position: Assistive Agents Need Accessibility Alignment
This paper argues that current AI agents are failing Blind and Visually Impaired (BVI) users because they are designed for sighted people who can easily verify information and recover from mistakes. The authors contend that accessibility should not be treated as a secondary feature or a simple interface tweak. Instead, they propose "accessibility alignment"—a new design framework that treats the specific safety, verification, and interaction constraints of BVI users as a primary requirement for any assistive AI system.

Why Current AI Agents Fail

The researchers analyzed 778 real-world assistance tasks and found that standard AI agents often struggle in assistive settings. These systems are typically built on the assumption that users can see the agent's work, easily correct errors, and tolerate trial-and-error. For BVI users, these assumptions are dangerous. When an agent provides incorrect navigation instructions or misreads a label, the user may not have a way to verify the information, leading to "silent failures" where the agent provides confident but wrong guidance. Because the consequences of these errors—such as physical injury or medical mistakes—are high, the current "move fast and break things" approach to AI development is fundamentally incompatible with the needs of BVI users.

The Four Core Stressors

The authors identify four environmental stressors that make assistive tasks uniquely challenging for AI:

• Limited Verifiability: Users often cannot independently confirm if the agent’s output is correct, making it difficult to catch hallucinations.

• High-Cost Errors: Mistakes in assistive contexts are often irreversible and carry severe real-world consequences.

• Cognitive Burden: BVI users often rely on audio or haptic feedback while navigating, meaning agents must provide information that is concise and well-timed to avoid overwhelming the user.

• Privacy Exposure: Assistive agents often capture sensitive data, such as private interiors or personal documents, requiring a higher standard of privacy management.

A New Design Pipeline

To address these issues, the authors propose a lifecycle-oriented design pipeline. This approach moves beyond just improving model accuracy and instead focuses on how an agent behaves under uncertainty. An accessibility-aligned agent must be able to recognize when it lacks sufficient evidence, communicate its uncertainty clearly to the user, and prioritize safety over efficiency. The authors suggest that by integrating these requirements into every stage of development—from initial user research to deployment and post-deployment feedback—developers can create systems that are truly reliable for BVI users.

A Call for Inclusive Design

The paper concludes that BVI-centered tasks serve as a critical "stress test" for all agentic AI. By solving for the extreme constraints faced by BVI users, developers can create more robust and trustworthy systems for everyone. The authors emphasize that accessibility alignment is not a byproduct of scaling up a model's general intelligence; it is a distinct objective that requires developers to rethink how agents communicate, manage risk, and handle their own limitations.