this post was submitted on 15 Apr 2026
1 points (100.0% liked)

Public Health

1720 readers
2 users here now

For issues concerning:


๐Ÿฉบ This community has a broader scope so please feel free to discuss. When it may not be clear, leave a comment talking about why something is important.



Related Communities

See the pinned post in the Medical Community Hub for links and descriptions. link (!medicine@lemmy.world)


Rules

Given the inherent intersection that these topics have with politics, we encourage thoughtful discussions while also adhering to the mander.xyz instance guidelines.

Try to focus on the scientific aspects and refrain from making overly partisan or inflammatory content

Our aim is to foster a respectful environment where we can delve into the scientific foundations of these topics. Thank you!

founded 2 years ago
MODERATORS
 

Led by Harvard medical student Arya Rao, a research team published in JAMA Network Open this week the results of a study that examined 21 leading off-the-shelf AI models in 29 standardized clinical vignettes. The bots all did fairly well when provided a full portfolio of medical information and asked to make a final diagnosis, with leading models correct 91 percent of the time. Early differential diagnosis, where clinicians try to rule out certain conditions while weighing various possibilities, is where that more than 80 percent failure rate comes in.

"Every model we tested failed on the vast majority of cases," Rao told The Register in an email. "That's the stage where uncertainty matters most, and it's where these systems are weakest."

In other words, it's the midnight anxiety-fueled WebMD rabbit hole of yesterday all over again, just supercharged with AI that's probably even more likely to get things wrong than you are without it.

top 8 comments
sorted by: hot top controversial new old

Jesus fuck, of course they do! Don't use a chatbot for medical advice!

There's neural networks specifically for diagnosis. Pattern recognition is kinda their whole thing, But a model trained on the whole internet has worryingly high odds of saying 'you have cancer of the butthole, LOL.' The correct incidence rate for that description in a medical context is never.

[โ€“] lakemalcom@sh.itjust.works 0 points 1 week ago (1 children)

For all models, optional real-time web search, browsing, and retrieval features were explicitly disabled when available. Each vignette was evaluated in triplicate. All replicates and vignettes were parsed independently. To ensure comparability, optional features such as real-time search were disabled across all models.

Hm. Seems like kinda hamstringing things

[โ€“] CorrectAlias@piefed.blahaj.zone 0 points 1 week ago* (last edited 1 week ago)

Maybe, but I think it's important to note that LLMs can hallucinate web results just the same. You can give them a specific web page and they'll sometimes spit out things that don't exist on the page, especially if you're doing it to correct a mistake the model made.

[โ€“] finallymadeanaccount@lemmy.world 0 points 1 week ago (1 children)

That robot has a long arm!

[โ€“] tacosanonymous@mander.xyz 0 points 1 week ago (2 children)

Inspector gadget style. My question is why does the robot have breasts?

[โ€“] P00ptart@lemmy.world 0 points 1 week ago

It's a fallout assaultron.

Because Elon wants to ultimately combine Optimus robots with Grok loli skins/AI programmed exactly how he wants it, so he can finally have someone who loves him for who he is, and not his money.

[โ€“] psycotica0@lemmy.ca 0 points 1 week ago

So you're saying there's a chance... ๐Ÿ˜›