Article Image

AI has a safety problem nobody is ready for

AI companies are racing to make their models safer, but those protections don’t appear to work equally well everywhere.

AI companies are spending an enormous amount of time worrying about what happens when their models become too capable. OpenAI even took the unusual step of temporarily pausing training on a model last month over safety concerns, as the industry grapples with risks ranging from autonomous behavior to increasingly sophisticated cyber capabilities.

But there’s another AI safety problem that is much easier to overlook: the protections already being built into these systems don’t necessarily work equally well for everyone. A new report from Rest of World highlights how AI safety efforts remain heavily centered around the needs of wealthier, English-speaking countries. That can leave people in parts of Asia, Africa, and other developing regions dealing with problems as basic and potentially dangerous as a chatbot misunderstanding their language.

Many of the biggest AI companies have extensive trust and safety teams, but the standards used to evaluate models are largely shaped in high-income countries. Researchers argue this creates blind spots when the same models are deployed in places with different languages, infrastructure, laws, and cultural expectations. Healthcare provides a particularly worrying example. Research into natural-language AI systems used in Africa has uncovered translation errors involving medical terminology. In Tigrinya, for example, machine translation has confused smallpox with syphilis and even translated “intravenous antibiotics” as “intravenous insecticides.”

The problem goes beyond translation. Research has also found that AI models can hallucinate more frequently in languages with relatively little training data. Safety guardrails that reliably catch harmful requests in English may also perform worse or be easier to bypass in those languages. That matters as people increasingly turn to chatbots for health information and other important decisions. It effectively creates a new AI divide: two people can use the same product and receive very different levels of protection simply because they speak different languages.

The timing makes this especially important because enthusiasm for AI is growing rapidly across many of these markets. China offers perhaps the clearest example. We’ve already seen AI adoption spread far beyond tech companies, with everything from schools to local governments embracing the technology. It has even reached Beijing dumpling shops, where one business began giving customers AI tokens and built AI-powered tools for managing queues. That enthusiasm isn’t necessarily a problem. But researchers warn that safety infrastructure needs to expand just as quickly.

Governments are beginning to respond. AI safety has featured in international initiatives including the Bletchley Declaration and this year’s India AI summit, while China has also proposed mechanisms to manage AI risks in developing nations. The larger challenge is making sure “AI safety” doesn’t only mean preventing a futuristic model from escaping its boundaries or launching a cyberattack. For millions of people already using these tools, safety can be much simpler: making sure a chatbot understands their language, recognizes when something is urgent, and doesn’t confidently give them dangerously wrong information. As AI spreads into everyday life, those problems are no longer edge cases. They’re part of what making AI safe actually means.

AI companies spend enormous amounts of time convincing us their technology is safe and secure. But what happens when those same companies become targets of increasingly sophisticated AI-powered attacks? Anthropic, the company behind Claude, is now facing that question. According to a Washington Post report, cybersecurity startup Cathedral discovered vulnerabilities that let its researchers access sensitive information in Anthropic's internal systems.

The researchers used AI agents to identify potential entry points in less than 12 hours. Interestingly, they didn't need to crack Claude itself. Instead, they exploited weaknesses in third-party business applications that Anthropic uses for everyday operations. Anthropic's security testing program disclosed the findings, which allows outside researchers to identify and report vulnerabilities.

Military working dogs charge into places the rest of us would flee. So, I understand why anyone would want to breed tougher, healthier dogs for the job.

Warhound, a new startup from Colossal Biosciences co-founder Ben Lamm and Anduril co-founder Palmer Luckey, has stepped out of stealth with $200 million to clone and gene-edit working dogs for American troops, police departments, and emergency services (via TechCrunch).

AI can describe a picture, identify objects, and answer questions about what a camera sees. But when a blind user needs a quick answer, getting an entire spoken essay about the scene isn't exactly helpful. Researchers at the University of Maryland (UMD) have found that AI-powered visual assistance can leave blind users doing too much work to get the information they actually need.

The study, titled Say It My Way: Exploring Control in Conversational Visual Question Answering with Blind Users, examined how people interact with Be My AI, an AI-powered feature of the Be My Eyes app. The researchers found that users often received lengthy descriptions that didn't adequately address their immediate goals, forcing them to ask follow-up questions or find ways to steer the AI towards a more useful answer.