I recently decided to give a fancy (i.e., paid subscription) new clinical decision-making AI tool a test run. I asked a simple question regarding specific guidelines I was already very familiar with. It quickly responded with incorrect information; with total confidence. Naturally, I called it out, and it folded immediately: “You were right to push back… here’s the accurate picture.”
Many of us are using AI on a pretty regular basis these days, and many use AI for health answers. So the question is, can we trust AI with our health decisions? The short answer is “no,” and that you should always follow up with your licensed healthcare provider. While AI can be a helpful tool to help you formulate better questions for your doctor, it can’t replace the years of clinical training, diagnostic reasoning, and personalized care of a human clinician.
Curious about what the stats actually show us? Keep reading…
When we look at the evidence, it becomes clear that AI’s medical advice is often “hit or miss.” A 2026 study published in BMJ Open evaluated five leading AI chatbots and found that half of the generated health answers were “medically problematic,” with nearly 20% deemed “highly problematic.” Interestingly, while responses were slightly stronger for well-documented subjects like oncology, they were most incorrect when it came to topics like nutrition (Tiller et al., 2026).
Why Do AI Chatbots Get Health Answers Wrong?
So now we know that chatbots are generating incorrect information, but why is it happening? The truth is that AI chatbots do not always pull their answers from vetted, peer-reviewed databases unless they are specifically programmed and instructed to do so.
Chatbots source a massive mix of internet content, such as blogs and forums, and do not necessarily obtain their information from peer-reviewed sources. They are trained on next-token prediction; given the text so far, they predict what comes next. Most of the time, the information lines up, but when it does not, AI cannot differentiate between correct and incorrect information. This means that what AI tells you is not necessarily based on what is factually correct, but what is most likely, which is also why AI “hallucinates.”
Can AI Make Up Its Sources Entirely?
Yep! And it happens more often than you may think when you use AI for health answers. That same BMJ Open study that we talked about earlier found that none of the five chatbots tested produced a fully accurate list of references. Citations were sometimes hallucinated outright or attached to the wrong paper, and overall, reference quality averaged a completeness score of just 40%. That’s a failing grade… and we are relying on them with our health.
If an AI gives you a stat or a study, ask it to name the actual source and authors, then take the time to check that the source is actually real and says what the AI claims. If it can’t produce a verifiable source, then it’s likely a hallucination, and you should look for other information sources.
Is It Unsafe to Use AI for Health Research?
Not necessarily unsafe, but it is definitely incomplete when used on its own. While it may provide good questions to bring to your healthcare provider, it becomes problematic when we use AI in place of a professional. A 2026 poll found that 32% of US adults turned to an AI chatbot for health information in the past year, most looking for a quick answer before deciding whether to see a provider. What was concerning was that 42% of people who asked AI about a physical health concern never followed up with a clinician, and that number climbed to 58% for questions regarding mental health. This can lead to delays in diagnosis and treatment, which is a concerning outcome.
What’s the Right Way to Use AI for Health Questions?
Use AI to build better questions, not to reach final answers. You can ask it to define an unfamiliar term, or put together a list of things to bring up at your next appointment.
The Bottom Line:
AI is a highly capable research assistant, but it is not a clinician. It has no internal clinical reasoning or safety-check system. It delivers every answer with the same confident tone, whether that answer is medically accurate or completely fabricated.
If you choose to use AI, use it to prepare for your next doctor’s visit, not to replace it.
Disclaimer: This content is for informational and educational purposes only and does not substitute for professional medical advice, diagnosis, or treatment. Always consult with a licensed healthcare provider regarding any health concerns.
