President Donald Trump announced on Tuesday that the White House is launching America.gov, an AI chatbot intended to help people find government services and other information.

Google confirmed that it is a partner on this launch and that its AI model, Gemini, is involved, but it’s unclear if other AI companies also contributed.

“Instead of forcing citizens to search through the endless maze of tens of thousands of government websites and rules… you’ll now have one front door for every single question,” Trump said in a post on X.

Making government services easier to navigate is a worthwhile goal, but is an AI chatbot the best way to do this? Large language models are not infallible and remain prone to hallucinations. In fact, CNN recently reported that the U.S. military nearly launched an armed operation against a Chinese vessel it believed was carrying components for nuclear weapons. The military aborted the mission at the last minute after realizing that the supposed threat was just an AI hallucination.

If someone receives faulty information from America.gov, it probably won’t trigger nuclear warfare, but a mistake can still have serious consequences. People might rely on this chatbot to access information about applying for food stamps, renewing their visas, or filing taxes, where errors could mean missed deadlines, denied benefits, or penalties.