AI: Assumed Imperfect, like people. Use it that way.

Do generative AI technologies, like ChatGPT, deserve all the abuse they get for "hallucinating" or just plain making things up?

Yes, but that shouldn’t stop you from playing with it, figuring out where you can apply it, and doing so, forthwith!

Look here:

This weekend, the New York Times ran a great story on “How Real People Are Using A.I. Right Now” — that link is a gift link from the small tranche the Times gives subscribers. Feel free to use the link and pass it around.

Everyone should read that article.

By the time you’re done with it, you’ll realize this “lying, confabulating, overhyped junk” is mighty powerful and practically useful. Just watch out for the sharp edges as you play.

After all, so what if it isn’t the great coming of a super-intelligent, fool-proof, silicon-based master race of machines preparing to enslave us all? (We already have them — they’re the people who are wrecking our social, economic, and governmental systems for their own benefit — but I promise this post is not about those issues, honest!)

There’s another article to read, on Medium, in a similar vein to the NYTimes article cited above entitled I’m an ER doctor. Here’s how I’m already using ChatGPT to help treat patients. Quelle horreur? (What a horror story?) A doctor using ChatGPT in the ER? From all we’ve heard of the limitations and risks of these Large Language Model (LLM) technologies, this doctor, Dr. Josh Tamayo-Sarver, would have to be irresponsible to do that, right?

Not so fast.

Doctor JT-S published another article in which he looked at using ChatGPT to assist in diagnosing patients’ ills in the emergency room (ER). Here it is and the results weren’t good.

“ChatGPT recently passed the U.S. Medical Licensing Exam, but using it for a real-world medical diagnosis would quickly turn deadly.”

Even more disturbing than doctors relying on ChatGPT to diagnose patients’ ills would be potential patients who self-diagnose their ills using ChatGPT and bypass the ER based on ChatGPT’s advice.

Here’s a key part of that article in the Doctor’s own words (I added my bolding for emphasis):

ChatGPT … misdiagnosed several … patients who had life-threatening conditions. It correctly suggested one of them had a brain tumor—but missed two others who also had tumors. It diagnosed another patient with torso pain as having a kidney stone—but missed that the patient actually had an aortic rupture. (And subsequently died on our operating table.) 

In short, ChatGPT worked pretty well as a diagnostic tool when I fed it perfect information and the patient had a classic presentation.

This is likely why ChatGPT “passed” the case vignettes in the Medical Licensing Exam. Not because it’s “smart,” but because the classic cases in the exam have a deterministic answer that already exists in its database.

ChatGPT rapidly presents answers in a natural language format (that’s the genuinely impressive part), but underneath that is a knowledge retrieval process similar to Google Search. And most actual patient cases are not classic.  

My experiment illustrated how the vast majority of any medical encounter is figuring out the correct patient narrative. If someone comes into my ER saying their wrist hurts, but not due to any recent accident, it could be a psychosomatic reaction after the patient’s grandson fell down, or it could be due to a sexually transmitted disease, or something else entirely. The art of medicine is extracting all the necessary information required to create the right narrative.

Might ChatGPT still work as a doctor’s assistant, automatically reading my patient notes during treatment and suggesting differentials? Possibly. But my fear is this could introduce even worse outcomes.

If my patient notes don’t include a question I haven’t yet asked, ChatGPT’s output will encourage me to keep missing that question. Like with my young female patient who didn’t know she was pregnant. If a possible ectopic pregnancy had not immediately occurred to me, ChatGPT would have kept enforcing that omission, only reflecting back to me the things I thought were obvious—enthusiastically validating my bias like the world’s most dangerous yes-man. 

So why I have enthusiastically suggested you play with ChatGPT (and similar LLMs), figure out where it can add value for you, and apply it there?

Doctor JT-S found in the article he published on Medium that ChatGPT, with supervision, did a great job converting medical-techno-jargon into easy-to-understand, tactfully communicated advice to patients and their relatives in emergencies. In the end, the doctor-author appears to have been behaving quite responsibly. Which is what you should do.

Don’t delegate critical responsibility to LLMs. You (and/or your employer) own the decision-making responsibilities. Consider the risks and act accordingly. But also evaluate the applicability and benefits.