Your favorite AI chatbot might not be telling the truth

By Bryan M. Wolfe Published March 11, 2025

AI search tools are becoming more popular, with one in four Americans reporting using AI instead of traditional search engines. However, here’s an important note: these AI chatbots do not always provide accurate information.

A recent study by the Tow Center for Digital Journalism, reported by Columbia Journalism Review, indicates that chatbots struggle to retrieve and cite news content accurately. Even more concerning is their tendency to invent information when they do not have the correct answer.

AI chatbots tested for the survey included many of the “best,” including ChatGPT, Perplexity, Perplexity Pro, DeepSeek, Microsoft’s Copilot, Grok-2, Grok-3, and Google Gemini.

In the tests, AI chatbots were given direct excerpts from 10 online articles published by various outlets. Each chatbot received 200 queries, representing 10 articles across 20 different publishers, for 1,600 queries. The chatbots were asked to identify the headline of each article, its original publisher, publication date, and URL.

Similar tests conducted with traditional search engines successfully provided the correct information. However, the AI chatbots did not perform as well.

The findings indicated that chatbots often struggle to decline questions they cannot answer accurately, frequently providing incorrect or speculative responses instead. Premium chatbots tend to deliver confidently incorrect answers more often than their free counterparts. Additionally, many chatbots appeared to disregard the Robot Exclusion Protocol (REP) preferences, which websites use to communicate with web robots like search engine crawlers.

The survey also found that generative search tools were prone to fabricating links and citing syndicated or copied versions of articles. Moreover, content licensing agreements with news sources did not guarantee accurate citations in chatbot responses.

What stands out most about the results of this survey is not just that AI chatbots often provide incorrect information but that they do so with alarming confidence. Instead of admitting they don’t know the answer, they tend to respond with phrases like “it appears,” “it’s possible,” or “might.”

For instance, ChatGPT incorrectly identified 134 articles yet only signaled uncertainty 15 times out of 200 responses and never refrained from providing an answer.

Based on the survey results, it’s probably wise not to rely exclusively on AI chatbots for answers. Instead, a combination of traditional search methods and AI tools is recommended. At the very least, using multiple AI chatbots to find an answer may be beneficial. Otherwise, you risk obtaining incorrect information.

Looking ahead, I wouldn’t be surprised to see a consolidation of AI chatbots as the better ones stand out from the poor-quality ones. Eventually, their results will be as accurate as those from traditional search engines. When that will happen is anyone’s guess.