Article Image

AI can describe a picture, but it still struggles to understand blind users

A University of Maryland study finds that Be My AI often gives blind users lengthy answers instead of the specific visual information they need.

AI can describe a picture, identify objects, and answer questions about what a camera sees. But when a blind user needs a quick answer, getting an entire spoken essay about the scene isn’t exactly helpful. Researchers at the University of Maryland (UMD) have found that AI-powered visual assistance can leave blind users doing too much work to get the information they actually need.

The study, titled Say It My Way: Exploring Control in Conversational Visual Question Answering with Blind Users, examined how people interact with Be My AI, an AI-powered feature of the Be My Eyes app. The researchers found that users often received lengthy descriptions that didn’t adequately address their immediate goals, forcing them to ask follow-up questions or find ways to steer the AI towards a more useful answer.

The researchers recruited 11 blind participants, who first tested Be My AI in a laboratory before using it in their daily lives for 10 days. The team analyzed 418 interactions, examining how participants asked questions, responded to the AI’s answers, and adapted their approach when the results missed the mark.

The results revealed a mismatch between what users asked and what the AI delivered. Participants typically asked questions of around 10 to 20 words, while the AI’s responses were more than 10 times longer. Conversations averaged three turns, but some stretched to 21 exchanges. Since the answers are read aloud, finding one useful detail can mean listening through a lot of unnecessary information first.

One example involved a blind participant checking whether sausages were cooked. The AI repeatedly reported that some parts still looked pink, but the user wanted practical guidance on what to do next, rather than another description of the food. In another case, a participant looking for her keys received a lengthy description of the image instead of a response focused on the missing item. More than half the participants considered the app’s habit of starting with a full image description a significant problem.

Participants tried several ways to get more useful answers. Some explained why they were asking a question, giving the AI context about their goals. Others broke complicated tasks into smaller questions. One participant checking bread dough asked separately about its size, surface, air bubbles, and flour, saying this was the only way to be sure the answers were correct.

While users found ways to steer the AI, the study exposed other limitations, including difficulty estimating distances and changes over time, inaccessible image framing, and limited guidance on camera positioning. The researchers argue that blind users shouldn’t have to master complex prompts to get simple answers. For AI visual assistance to be truly accessible, it must focus on what users need to know and deliver clear, useful answers without making them work around the technology’s shortcomings.

Because when someone asks where their keys are, they probably don’t need a guided tour of the entire room.

Military working dogs charge into places the rest of us would flee. So, I understand why anyone would want to breed tougher, healthier dogs for the job.

Warhound, a new startup from Colossal Biosciences co-founder Ben Lamm and Anduril co-founder Palmer Luckey, has stepped out of stealth with $200 million to clone and gene-edit working dogs for American troops, police departments, and emergency services (via TechCrunch).

Platforms tend to shrug off AI clutter until it threatens what they sell. For Etsy, that thing is trust in the maker.

To protect that, Etsy has banned AI-generated how-to guides and listings made entirely by bots. Most sellers must show at least one unretouched photo of the item (via Etsy Seller Handbook).

Forget smartwatches. Scientists have developed a touchscreen that can be woven directly into clothing, letting users interact with digital content without carrying a rigid display. Researchers at the National University of Singapore (NUS) have integrated 54 specialised fibres into a sleeve to create what they describe as the world's first touchscreen built into a garment. Published in Nature, the research could open the door to clothing that does more than just look good. Your next smart outfit might actually be smart enough to show what's happening on your phone.

Your clothes could soon run apps