"AI hallucinates because it’s trained to fake answers it doesn’t know" Celina Zhao

Sam: M foreign.

Narrator: AI hallucinates because it's trained to fake answers it doesn't know so, wrote Selina Xiao in 2025 in an article published in the journal Science. If you've ever used any kind of AI tool like ChatGPT or Copilot, you may have come across errors in answers that are stated as fact. This could be a result of what has become termed as AI hallucination. So what exactly is an artificial intelligence hallucination? There is no one definition, but we could consider it as moments when an AI technology produces answers that are not merely wrong, but wholly fabricated or fake. The AI creates facts or scenarios that do not actually exist in reality or are, uh, not necessarily present in the data it was trained on. A more technical way to consider AI hallucinations could refer to when AI generates confident and plausible but factually incorrect or unverifiable information. AI models often use probabilistic text prediction rather than truth validation to produce answers. The answers AI produces in these circumstances are could be considered fake. AI hallucinations occur in large language models because these systems predict likely word sequences for answers rather than retrieving verified facts, making large language model hallucinations a, uh, structural byproduct of how generative AI models function. Because AI models predict what is likely required in an answer, not what is true, they may often generate inaccurate or fabricated information. AI hallucinations are therefore not a glitch. They are simply a natural outcome of how these AI systems are designed. But AI hallucinations matter because the answers produced could reduce reliability, introduce misinformation, and create risks across domains such as legal, medical and search systems, all of which may have cases where accuracy is critical. Understanding why these hallucinations occur is essential if we are to trust AI systems and deploy them responsibly. In fact, as referenced in the financial times, in March 2026, a survey of 80,000 plus users of Anthropic's Claude tool claimed that users of AI say their biggest concern is not being replaced by the technology, but its propensity to make mistakes. Another area where hallucinations could be problematic is where they undermine scientific research. For example, at UH, one of the world's top academic AI conferences, research papers were accepted with 100 AI hallucinated citations. This type of fabrication could present problems for ongoing research. More specifically, the occurrence of AI hallucinations could pose significant concerns and challenges in several domains. In healthcare, for instance, a machine learning model could misdiagnose a patient based on fabricated data, leading to potentially harmful treatment plan suggestions. Similarly, in legal contexts, an AI system might generate false evidence or testimonies, compromising the integrity of judicial proceedings. Financial systems are also at risk. An AI driven trading algorithm could make investments based on inaccurate market predictions, resulting in substantial financial losses. The implications of AI hallucinations extend beyond these high stakes environments. In everyday applications such as chatbots and virtual assistants, inaccuracies can lead to misunderstandings and dissatisfaction among users, undermining the trust in these technologies. The broader societal impact includes the potential for misinformation spreading through automated content generation tools or social media bots that lack a reliable fact checking mechanism. Therefore, understanding the reasons behind AI hallucinations is crucial not only for mitigating their adverse effects, but also for harnessing artificial intelligence's full potential responsibly and ethically. At its core, an AI model learns by adjusting internal parameters until the predictions it makes from a training data set match known outcomes as closely as possible. Models are designed to always produce an answer rather than say, I don't know. But this answer will be based only on the limits of its training data. However, this can produce a phenomena called overfitting. Overfitting can manifest in several ways. One common sign is when a model performs exceptionally well on its training dataset but underperforms significantly on new data. This discrepancy indicates that the model has memorized the training examples rather than learning generalized principles. For instance, an image recognition model trained solely on pictures of cats may become proficient at identifying cats but fail to recognize other animals in new images. Overfitting is akin to a student who can recite every line of a textbook but fails to answer any question that deviates from the exact wording of those pages. Another problem is that of data bias. No AI is immune to the biases already present in its training data. If the data set reflects historical prejudices or uneven representation, the model will internalize and amplify those distortions. A well known historic case involved Amazon's recruiting system, which penalized female applicants because it was trained on application forms that historically favored men. The model's bias was not a flaw of its architecture, but a mirror of past hiring practices. The impact of such biases can be profound and far reaching. In healthcare, for example, AI systems that diagnose diseases may provide different treatment recommendations based on the patient's race or socioeconomic status, leading to disparities in medical care within populations. Similarly, in criminal justice, biased algorithms used for predicting recidivism rates could result in unfair sentencing and parole decisions. Beyond bias, sheer scarcity or incompleteness of data can drive Hallucinations. A UH model trained on a narrow slice of information, say medical records from a single hospital, will lack exposure to variations in patient demographics, treatment protocols, and disease presentations from other hospitals. Consequently, when confronted with new cases, the AI could fabricate explanations that fit within its more limited training data set. Stale or outdated data further aggravate the issue. Language evolves. Cultural references shift. Scientific knowledge advances. If an AI system continues to rely on a frozen data set, it can generate responses rooted in old information. In finance, for example, a trading algorithm built on historical market conditions may misread current signals if it has not been retrained with recent data. To address these challenges and mitigate the risk of hallucinations, several strategies can be employed. One approach is to expand and diversify the training data sets by including a broader range of examples that represent different demographics, contexts, and scenarios. This helps ensure that the model learns more generalized patterns rather than becoming overly specialized in specific instances. Future research promises to further reduce hallucination rates. Advanced data augmentation can add to the size of the underlying datasets by synthesizing data that are realistic examples of real world data. Other research is being done on adaptive learning systems that update their knowledge base in real time and may better cope with shifting contexts, such as those involving new slang, new medical treatments, or fluctuating market conditions. AI models are increasingly designed to more carefully reason step by step via a UH process known as as chain of thought. This takes longer, but it can reduce hallucinations. Mitigating hallucination risks requires continuous data curation, actively adding fresh examples, ensuring balanced representation across categories, and periodically purging outdated entries. Continuous evaluation mechanisms that monitor AI model performance automatically could serve as early warning signs. By detecting subtle drifts in accuracy or bias, these automated systems could trigger retraining before hallucinations become more pervasive. From another perspective, hallucinations may sometimes be considered useful. Hallucinations may show you a different angle to a problem that you've never thought about, or the answer that is provided to fill an information gap may actually be a alternative approach that you haven't thought of before. This could shift your perspective. So embracing hallucinations is possibly a different way of seeing the world, as opposed to simply considering answers wrong based on available evidence. In conclusion, AI offers unprecedented capabilities, yet its power is double edged. Hallucinations can arise from overfitting models, biased data, and inadequate training. This can all highlight the fragility of systems that learn from imperfect information. By acknowledging these vulnerabilities and systematically addressing them through robust engineering practices, transparent governance and ongoing interdisciplinary dialogue, we can harness AI's benefits while safeguarding against its missteps. In doing so, we ensure that the future of AI is not one of illusion, but one grounded in reliable knowledge, ethical responsibility, and human trust. What should be clear is AI doesn't hallucinate in the human sense. Instead, AI can fabricate and make things up confidently and convincingly. These hallucinations may take the form of fake facts, invented quotes, incorrect citations, or completely fabricated people, places, or events. Sometimes they're harmless. Sometimes they're dangerous. Always they raise important questions about how much we can or should trust AI. Thank you for listening, and we hope you will find interesting other episodes of the AI Future Podcast.

Sam: Sam.