AI helped a very close family member with a complex gastrointestinal chronic condition that doctors in three countries could not resolve. Not because of a novel diagnosis or some hidden insight, but because it reduced cognitive load, provided accessible, detailed coaching, and restored agency. That, in turn, recruited the uniquely human placebo effect.
That’s the line worth defending.
“Health” modes in LLMs are not new. Google has had Med-PaLM for years, alongside many other systems that never made headlines. What is new is that three very different categories of health AI are now being marketed and conflated.
First, generalist health chatbots.
Broad reasoning models with medical guardrails. Their strength is not diagnosis, but synthesis, explanation, and longitudinal sense-making. In messy, real-world cases, that support can be genuinely valuable. I’ve seen this first-hand. The benefit wasn’t certainty. It was orientation.
Second, narrowly trained medical models like Med-PaLM.
Built by companies such as Google, Epic, IBM Watson Health, often embedded quietly inside hospital systems.
These models excel at constrained problems: guideline adherence, protocol checking, triage support, coding, and clerical reduction. This is why they matter in acute care, imaging, oncology workflows, and hospital operations, where small accuracy gains compound into real clinical and financial impact.
Where they struggle is not intelligence, but scope. Longevity, preventive care, and complex chronic conditions are not clean classification problems. They involve weak signals, trade-offs across systems, long time horizons, and incomplete data.
These models are necessary infrastructure, but not sufficient intelligence.
Third, autonomous or semi-autonomous clinical systems.
Systems that ingest patient data, route workflows, and trigger actions using a mix of deterministic rules, narrow models, and general LLMs, depending on the task. This is my area of work with Kalibra.
Across all three categories, the choke point remains the same: data ingestion. Health data is overwhelmingly fragmented and unstructured. PDFs, portals, screenshots, wearables, narratives. Without a structured, longitudinal data layer, even the best model is guessing politely.
So why do we need physicians more than ever?
Two reasons. First, accountability and trust. Human clinicians uniquely interpret context, align decisions to values, and bring empathy and intuition to the interaction layer.
Second, liability remains unresolved by design. When AI advice influences behaviour and harm occurs, responsibility snaps back to humans. No vendor is absorbing that risk at scale.
The real question is who controls the intelligence layer between raw data, AI reasoning, and human decision-making. If physicians don’t lead that layer, platforms will. You already know what happens next.
Further reading: The AI Mirage: Why Healthcare Can’t Run Before It Crawls · How Generative AI Will Change Your Health Journey · The Heart Outside the Body, the Brain in the Cloud
