TL;DR
Several governments have been publicly embarrassed by AI hallucinations in official reports over the past two years. These incidents reveal vulnerabilities in AI use and raise questions about verification processes.
Multiple governments have publicly acknowledged and faced embarrassment due to AI-generated false citations appearing in official documents over the past year, highlighting risks associated with unverified AI use in official policymaking and reporting.
In April 2025, South Africa withdrew its Draft National Artificial Intelligence Policy after discovering at least six AI hallucinations in its bibliography, including fictitious academic sources, as confirmed by civil rights group Article One. Minister Solly Malatsi attributed the error to unverified AI citations, emphasizing accountability measures.
In the United States, a May 2025 report on children’s health, titled ‘Make America Healthy Again,’ contained numerous incorrect references, including nonexistent studies and misattributed authors. The White House labeled the errors as ‘formatting issues’ and issued a corrected version shortly after.
Australia faced scrutiny in August 2025 when a Deloitte report for the Department of Employment and Workplace Relations was found to include fake references and made-up quotes, leading Deloitte to acknowledge AI-induced inaccuracies and refund the government $290,000 of the original $440,000 fee. Similarly, in Canada, Deloitte’s healthcare report for Newfoundland and Labrador contained fabricated citations, prompting a re-release after corrections and policy updates on AI use in government contracts.
Europe’s cybersecurity agency ENISA admitted in 2025 that two of its threat reports contained 26 incorrect footnotes out of 492, stemming from AI hallucinations. Experts criticized the agency’s lack of verification protocols, raising concerns about reliance on AI for authoritative publications.
Why It Matters
These incidents underscore the vulnerabilities in current AI tools when used in official government contexts, risking misinformation, policy errors, and damage to institutional credibility. They highlight the urgent need for rigorous human verification and clear AI governance policies to prevent future embarrassments and ensure accountability.
As an affiliate, we earn on qualifying purchases.
Background
Over the past two years, AI hallucinations—instances where AI generates false or misleading information—have increasingly appeared in official documents. Governments and organizations have struggled with integrating AI responsibly, often relying on unverified outputs. The South African policy incident marks the first government withdrawal due to AI hallucinations, but similar issues have emerged globally, including in the US, Australia, Canada, and within European cybersecurity bodies. These cases reflect broader concerns about the unchecked use of generative AI in critical decision-making and reporting processes.
“There will be consequence management for those responsible for drafting and quality assurance.”
— Solly Malatsi, South Africa’s Minister of Communications and Digital Technologies
“The use of generative AI tools resulted in inaccurate outputs whereby certain citations contained errors.”
— Deloitte spokesperson (Australia)
“ENISA let AI touch the one layer it should never touch unguarded: the truth layer.”
— Researchers from Westfälische Hochschule cited by Der Spiegel
AI citation verification software
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
What Remains Unclear
It remains unclear how widespread AI hallucination issues will become as governments increase reliance on AI tools. The long-term impact on policy and credibility is still uncertain, and ongoing efforts to establish verification standards are in early stages.
Tool Users Terminal Mug – AI Output Sanity Check Design – 11 oz Ceramic
- Design Theme: AI Output Sanity Check Checklist
- Print Style: Double-Sided Print for Visibility
- Capacity and Material: 11 oz Ceramic Mug
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
What’s Next
Next steps include implementing stricter verification protocols for AI-generated content, developing clear guidelines for AI use in official documents, and monitoring the effectiveness of these measures. Governments are likely to increase oversight and transparency around AI applications in policy and reporting processes.
As an affiliate, we earn on qualifying purchases.
Key Questions
How common are AI hallucinations in government documents?
While these incidents are increasingly reported, they remain relatively rare but are growing as AI tools become more integrated into official workflows. The recent cases highlight the need for better oversight.
What measures are governments taking to prevent future AI hallucinations?
Some governments are introducing verification protocols, updating procurement policies to include AI risk assessments, and demanding human review of AI-generated content before publication.
Could AI hallucinations impact national security or policy decisions?
Yes, if unchecked, false information from AI could influence policy, misinform decision-makers, or undermine trust in government institutions, emphasizing the importance of strict oversight.
