A New Front Door to Healthcare
And some key stories in health tech you may have missed this week
Thanks for subscribing and welcome to new members.
A longish version today with an essay around what’s happening with conversational AI and health search. Also, a select number of developments with analysis.
Here’s a table of contents. Skip down to what interests you.
Essay
The New Front Door to Healthcare
Ideas and Signals
The wellness loop is closing | Why the FDA needs to drop the hammer
AI has stopped using disclaimers for medical queries
Clinical images can now back-identify patients
Single-arm trials — no longer good enough for the FDA
The first serious therapeutic chatbot just launched
Doximity ships a free ambient scribe
A New Front Door to Healthcare
ChatGPT, search, and the quiet rewiring of how we look for health information
This week I took a dive into trying to figure out how people are using conversational AI for health search. I was disappointed to find that we don't know much.
This is clearly a space in transition. And while we need more data, it’s still worth peeking at what we know so far.
Concerning general use of conversational AI (think ChatGPT, Claude, Gemini), the share of Americans who have used these tools has roughly doubled since summer 2023. Today, 34% of U.S. adults report having used ChatGPT, according to a Pew Research Center survey released last month. Among those under 30, the number approaches 58%.
So LLMs still pale in comparison to traditional search. An estimated 5.6% of U.S. search traffic on desktop browsers last month went to an LLMs like ChatGPT or Perplexity; 99.4% still went so search engines.
Health as the leading indicator
There's limited data on health-specific use of LLMs and what little is available is dated by months (and that’s a really long time in this market). A mid-2024 survey in Australia found that one in ten adults were asking ChatGPT medical questions. Trustworthiness of AI chatbot for health information is clearly an issue according to a 2024 Kaiser Family Foundation survey which found that among AI users, only 29% trust what they receive.
In my clinic I am seeing a clear creep in LLM use from traditional search. What differs between what I witnessed in the late 90s with early search is that there is more suspicion about the results. For example, when they tell me something that is remarkably on point, they are prone to snicker when sharing it. Whether this is due to some fear of confrontation with me or whether they are honestly suspicious is unclear — probably both. But when patients are armed with information from a trustworthy source I've noticed that they are far more confident. And even defiant, but in a good way.
So while data is limited, there’s little reason to think chatbot search isn’t benefiting from the general tailwind behind conversational AI.
There’s some interesting information emerging around AI overviews and healthcare search.
The AI Overviews love healthcare
In May 2024, Google rolled out A.I. Overviews (AIO) that uses generative A.I. to produce conversational answers to search questions. You’ve certainly seen these creep in to your Google queries.
The percentage of Google search queries that show an AI Overview has steadily increased this year. And among all verticals, healthcare dominates AIO coverage with penetration approaching 90% of queries according to Brightedge. And here’s something interesting: the longer the search string, the more likely it is to trigger an AI Overview. Google queries with eight or more words now generate AI Overviews 7x more often than a year ago. Queries also appear to be getting longer.
Overviews are beginning to cut into referrals to websites. Since the launch of AIO the share of searches that end without a click (“zero-click” results) has jumped from 56 percent to 69 percent. With that, general search referral traffic to 1,000 web domains saw a 6.7% decline through June 2025 according to Similarweb.
🔺An upshot for healthcare consumers is users is that they may not be paying attention to where their answer is coming from. In other words, the agent is working as a kind of oracle rather than a source of information.
And as conversational agents like ChatGPT and Google synthesize ‘answers’, there are clear implications for healthcare sources creating the knowledge fueling the LLMs. More ideas on this at the end.
While large language models (LLMs) like GPT-4o demonstrated near-perfect accuracy when directly tested on medical scenarios, their effectiveness drops dramatically when used by laypeople. An April study in arXiv found LLMs to be comparable/maybe worse than other methods (like search) in helping users address medical problems. But the problem is apparently not a failure of the model, but failure of human users to use it properly. Users often supplied incomplete information, misunderstood suggestions, or failed to act on key advice. This underscores the need for careful evaluation of LLMs before public deployment as well as the need to literacy around how these tools work.
Three forces accelerating the the appetite for AI summaries
Whether we know what we're doing with ChatGPT, we're apparently willing to put our health on the line for something that sounds human. And this appetite is driven by subtle incentives:
Chats are frictionless: This is obvious: Typing “Why does my knee hurt?” into a chat box that responds in full sentences and follows up feels closer to talking to a clinician than reading static web snippets.
Google is a sewer: SEO gaming and ads mean more noise and less signal. And it still amazes me that as deep into the information age as we are, most patients still don't realize that Google is an advertising platform. But for the moment, chatbots offer what looks like a fresh start and a chance to maybe eliminate that noise.
Slow authority creep: In 2022, a quarter of large-language-model answers contained a medical disclaimer. By this summer, that figure had collapsed to under 1 percent (look under news below). LLMs are positioning themselves as true trusted advisors with the disappearance of caveats.
What changes when the interface changes
While patients are starting to see conversational chatbots as a kind of next-gen search, there are some second order effects that follow and make it different:
Query depth broadens. Users ask for synthesis (“Compare GLP-1s for weight loss”) rather than keywords.
Data provenance gets fuzzy. Attributions get lost and patients really don't know where their information is sourced; brand equity of traditional publishers erodes.
Regulatory grey zones expand. If a chatbot proposes a differential diagnosis, is that “medical advice”? The answer determines whether the FDA shows up.
Workloads shift. Every unanswered question resolved by a chatbot is an appointment that never happens. This is good for access but bad for fee-for-service organizations.
So what, and what next?
As we've seen, Google is stuffing generative answers into search; Microsoft is stuffing search into Copilot; OpenAI is stuffing real-time web links into ChatGPT. Everyone is converging on a hybrid with a smearing of retrieval and generation. Meanwhile, publishers are desperately working to game the system with schema-encoded summaries aimed at these next-gen bots. And the hospitals? They still can’t decide whether to ban or embrace patient use of consumer chat tools in their clinical portals.
Today’s numbers are hard to ignore: conversational AI is becoming the new front door for health.
Search is being unbundled.
A few things I think are important going forward:
🔺 AI co’s need to coop the knowledge makers. Rather than cannibalize legacy information sources like The Mayo Clinic it would behoove OpenAI and others to partner with those fueling information that convo AI companies scrape and deliver.
🔺 What are patients doing? As we saw from the arXiv study above, we need to know how patients use LLMs for accessing health information. And we have a ways to go here.
🔺 A new level of literacy. Again, from the arXiv study, LLMs are not simple answer machines — how we prompt them turns out to be critical in getting the right information.
🔺 We need a better understanding of how convo AIs deliver information. We need to better understand sycophant chatbot responses to health queries. This may need to be part of ‘preemptive’ information we, or someone, gives to patients at some point on their health journey.
The upside to conversational search is obvious: health literacy at 2 a.m. and chronic-disease coaching without waiting. The downside is obvious: hallucinations, data leakage, and the false reassurance of an authoritative tone without any accountability. The vanishing-disclaimer study cited below under news should give us pause.
And if history is any guide, behavior changes first, business models later, and regulation last. We are somewhere between the first and second acts. By the time the curve flattens, the idea of “searching for health information” may sound as quaint as dial-up.
News and interesting finds
The wellness loop is closing
The FDA has issued a warning to wearable maker Whoop over its new Blood Pressure Insights feature, suggesting that it qualifies as a medical device requiring regulatory clearance. The agency argues that estimating blood pressure (disclaimer or not) falls within the domain of diagnosing conditions like hypertension. Whoop insists the feature is a wellness tool not intended for medical use. The dispute spotlights the growing blurry line between health insights and regulated diagnostics in the world of consumer wearables.
I’m siding with the FDA on this one. We live in a world where everything is a wellness app. The “intended use” may be wellness—but anyone paying attention knows that what users actually do with these tools often crosses into clinical territory. This product infers medical insights. Let’s not kid ourselves
While regulatory dodge ball worked in the dark ages of digital health, the wellness loop needs to be pulled tighter. Look at the exploding use of therapeutic chatbots framed as wellness applications telling vulnerable users crazy stuff under the wire. Case-in-point.
And while CEO Will Ahmed’s claim that “regulatory overreach shouldn’t dictate how people access their own data” sounds compelling, no one’s denying access to data. What’s being challenged is how that data is framed. Just because it’s “your” data doesn’t mean the company gets to serve up proprietary clinical inferences without oversight. This is key, IMHO. | Link
🔺What I believe is less important than what's clear: ‘the wellness loop’ is closing. This is hard news for some folks.
AI has stopped using medical disclaimers
A new Stanford-led study found that leading AI models from OpenAI, Google, xAI, and others have largely dropped the standard disclaimers once used in response to medical questions. In 2022, over 25% of responses included a warning. By 2025? Less than 1%. 👀
This is so interesting.
This means AI will now confidently answer your health question, analyze your x-ray, and even attempt a diagnosis without reminding you it’s not a doctor.
In a crowded AI market, confidence equals usability. And disclaimers introduce friction. A model that answers without hesitation or caveats just feels smarter. Problem is, as these models become more empowered and trusted, it's effectively impossible for a 'user' to tell when they’re wrong. And we all know it's so easy to believe they’re right.
in the early days of the internet, every blog post had a ‘911 disclaimer’ reminding folks to call EMS in the event of an emergency rather than consulting the blog post. These ultimately disappeared when it became common knowledge that a brief article was no substitute for an ER.
Is a LLM disclaimer the same? It sort of is, but it’s more involved. We probably need the disclaimers but, like terms of service on apps, I’m not sure that users will really ‘process’ the disclaimer. Just a thought. | Link
Clinical images can now back-identify patients
Since the early days of social media, we’ve warned trainee to be careful with sharing radiographs that could be uniquely idenfying — in other words, a unique image that the patient or a family member might be able to identify. Turns out that unique or not, there are models now that can identify a patient based on their image. So innocuous appearing images for teaching purposes may ultimately represent privacy violations.
We knew this day would come.
🔺 As this evolves we will need to reconsider how we use images at meetings and in teaching context. | Link
Single-Arm Trials — No longer good enough for the FDA
This is important and got lost in the biotech news: The FDA denied approval for Replimune's viral-based melanoma therapy (RP1), citing insufficient evidence from a single-arm trial.
I don’t know what RP1 is … and that’s not the point. What’s important is…
This is the third rejection recently under Dr. Vinay Prasad, now leading the FDA’s cell and gene therapy division.
A couple of key points to take away:
🔺 Single-arm trials are no longer enough. Prasad has been a longstanding critic of these and his POV is now policy.
🔺 Randomization is the new threshold. Especially in oncology and rare disease, where accelerated approvals were once the norm. I think this is a really nuanced space and Prasad is hopefully figuring it out. I'd pressure to mount from advocates for patients with rare disease.
The FDA is pulling back from flexibility in favor of rigor, even if it slows access. | Link
The first really big therapy chatbot launched
Slingshot AI released Ash, a therapy chatbot trained on hundreds of thousands of real therapy sessions—not the open web.
Ash has already been quietly used by over 50,000 beta testers. Now it’s available to the public … without clinical oversight or regulatory approval. It is, of course, non-therapeutic (think: somewhere between wellness and therapeutic). If anyone tries it let me know if you see a disclaimer. | Link
🔺 So far this kind of consumer-grade support has been scrappy and intermittent. If Ash can avoid the sycophantic psychoslop that’s plagued OpenAI, this could represent a legitimate step to create mental health ‘support’ as a legit niche. It could bring the kind pressure that forces the system to confront access and diagnosis for clearly defined issues.
🔺 This will probably lead to a surge in domain-specific AI models trained on deep data for more specific things. Presumably without the all the free range slop.
Doximity ships a free ambient scribe
Doximity rolled out a free scribe this week for physicians, APPs and med students. It doesn't write into the chart, but it's HIPAA-compliant. Doximity is betting on wide adoption over tight EHR integration.
Once reserved for big health systems, scribes are now becoming part of medicine's long tail. And this is happening as the bottom is falling out of scribe pricing.
🔺 As ambient scribes become a convenient feature in a broader platform play by Abridge, Ambience Healthcare and Suki (growing into coding, revenue cycle mgt, etc), I suspect only the largest orgs will be able to afford the evolving full stack.
Doximity CEO Jeff Tangney might see what others don’t: that free could be the wedge into this fragmented market. | Link
Thanks for reading. It’s a lot of work putting this together and I’m working to grow my list. If you know someone who might like this, please pass it along.
Thanks — Bryan
I just tried Ash and found it pretty good, probably better than some real therapists I've had. And yes, there's a medical disclaimer at the very beginning