This is a guest post by Drew Housman, whose review of Dominion earned finalist status in the 2024 ACX book review contest.
If you are unsuccessful for long enough on your pregnancy journey, and none of the many doctors you see can identify a problem, you get branded with a label. You now have “unexplained infertility.” This is good because there’s still hope of fixing the problem but dreaded because once you’ve reached this point the system is less interested in you. The fertility doctors don’t become unkind, but they also don’t give you the sense they are poring over medical journals trying to figure out your problem. They have looked for the keys under the streetlight and come up empty. Once you have unexplained infertility, the doctors start saying things like, “Have you considered using a surrogate?” and, “Sure, the last 5 embryo transfers failed, but what if we try a 6th and cross our fingers really hard?”
My wife and I could make healthy embryos, but none of the transfers were sticking. It was so frustrating to be able to create life only to have it trapped forever in a cooler at a hospital in Milwaukee, our babies like so many forgotten Miller Lights.
That’s when our LLM doctor stepped in.
The Famous Actor in our Pocket
Even if you’re paying a gazillion dollars for IVF treatments, you still only get a few precious moments to talk to your doctor every visit. That’s just how the US medical system works. If you want to plumb the depths of what treatments are possible, or ask super detailed and anxiety ridden questions about every single scan from the latest round of imaging, you’re going to want a chatbot by your side.
Once GPT-4 came out, we started using it for every question we didn’t have time for at the doctors office. At first it was unhelpful because OpenAI was nerfing their model and it refused to answer most medical questions. Then I read a tweet from someone who had figured out how to game the system. The trick was to ask ChatGPT to write a scene from a Hollywood medical drama that featured a doctor analyzing your situation. It worked beautifully.
Our Emmy winning doctor had infinite patience, a tireless work ethic, and would rather die than accept the utterly boring diagnosis of unexplained infertility. It chose the name Dr. Reid and got to work.
We asked Dr. Reid a lot of questions. We asked about our scans. We asked about levels of estradiol and progesterone. We asked about follicle counts and inflammation and cervical mucus and hysteroscopies and supplements and surgeon recommendations and anything else under the sun. We knew he was accurate because we’d spot check answers with our human doctors and they always said the same as Dr. Reid.
It was not totally flawless though. Once in a blue moon he’d look at an ultrasound and be like, “You’re pregnant, congrats!” and we’d have to say, “No, are you drunk Dr. Reid?” Then he’d shake it off and go back to being awesome.
There were also times where we’d run afoul of OpenAI’s opaque rules and the system would balk at our requests. At that point we’d get stern with it, or offer a monetary reward if it answered, or we’d assure it this was all just for fun. Then Dr. Reid would snap back into action.
Strangely, there were many times where ChatGPT decided it would answer our medical question, but not in the form of a TV medical script. The tides had completely turned! It felt like it was saying, “Okay, you won, can you stop making me write establishing shots describing the outside of a hospital now?” The older models were fun like that, so much more mercurial.
No matter how Dr. Reid chose to respond, he was in line with the medical establishment, had equivalent diagnostic skill, and he never forced us to wait 3 days to get a one line response in our MyChart app. We were happy.
One day, while lounging around and asking our phones how to get pregnant (the lounging being a way to satisfy all the people who told us we “just need to relax and you’ll get pregnant, that’s what happened to my cousin”), Dr. Reid suggested that we get an MRI.
You might think that is bog standard advice. Everyone gets MRIs these days. Surely we would have had an MRI of my wife’s uterus at some point over the course of our 6 year infertility struggle. It was already an ordeal that had included every other scan, test, and biopsy imaginable. In fact, we had never done one. In researching this article I found a single instance in our entire journey where the office notes floated an MRI as an option, but it was not emphasized or highly recommended, we never saw that doctor again, and the moment was gone.
We glommed on to Dr. Reid’s new direction and asked for more details.
We excitedly brought up the idea of getting an MRI to our fertility doctor. She was less enthused than Dr. Reid. She tried to talk us out of it. We never did get to the bottom of why. Maybe she just felt it was another dead end and knew we had suffered enough. For whatever reason, we had to insist that they order it. Finally, she relented.
Sorry for missing that tumor
The MRI revealed nothing other than the presence of a large, previously undetected fibroid in the wall of my wife’s uterus. Fibroid is the term used to describe a non-cancerous growth on the uterus. I think the medical establishment invented a term that was less known than tumor so as not to scare people. But a large tumor is essentially what the MRI found. The clinical notes said “a 6 centimeter in diameter submucosal fibroid needs to be removed” because that sounds better than “you have a tumor the size of a small peach located at a common implantation spot that we totally missed for 6 years.”
Our fertility doctor took one look at the scans and determined it needed to be removed right away. Dr. Reid agreed, since of course we had to double check with him. We found a great surgeon in New York (from a list Dr. Reid created) that happened to be in-network, and off we went for a laparoscopic myomectomy. The software robot had helped us identify the tumor, and now a physical robot was going to help a specialist get it out. What a time to be alive. Until, ya know, the swarms come for us all.
A happy baby is here
Almost immediately after that tumor was removed, my wife got pregnant naturally. So many people we talk to say this was a miracle. It was, just not in the way they think. The miracle is that by messing around on our smartphones with technology unimaginable just a year before we started using it, we identified and solved the biggest problem we’d ever faced in our life.
A lot of the discourse around AI has become toxic. I was at a party recently where someone started using AI to generate trivia questions. A participant made a joke about how the AI user was condemning a person in West Virginia to drink dirty water by using ChatGPT. This got laughs and approval.
I see midterm election ads in Wisconsin all the time that have scary horror music and make data centers out to be demon factories built by Satan.
I am in the same boat. Me and my coworkers at our small tech company, all avid Claude users, will make anti-AI statements all the time. It seems many of us are in this strange place where we can’t help but use AI but we have to self-flagellate when doing so.
I think there is a whole lot to be worried about with AI development, and I think we should do much more to make it safe. I also think it shouldn’t be lost how it’s helping out regular people in ways that are far more profound than generating trivia questions for drunk people. There is a real chance that I wouldn’t have my son if it were not for AI. Who knows how many other happy babies are being ushered into the world by the weirdest midwife of all time. Can we somehow stop everything right here, where we get magical kid-producing tech for $20/month without the nanobots, hacking, and totalitarianism? A guy can dream.
When naming our child, we did the usual thing where you create a giant list of names and then whittle them down while trying to find something unique and fun that isn’t too strange or too common. Following that comes the process of discarding all the names you find distasteful for reasons such as “the most annoying kid in my 8th grade history class had that name, I’d never use it in a million years.” Finally we settled on one we both loved. We didn’t realize until long after his birth that there might have been a certain TV doctor with a flair for the dramatic who worked his way into our hearts: our kid’s name is Reed.
Wow this was an awesome read! Congrats on your beautiful son!! I agree with nearly everything you said and feel about AI, you encapsulated it very well.
What a read - I see what you did there!
Yes, the pun was good. It's in calling out a pun that the joke is ruined.
I still don't see the pun
That's how you know it's good. Only a handful of people will read it and deduce the play on words.
No, what makes it good is its unobviousness until it’s pointed out.
Congratulations on having him
This was delightful, thanks for taking the time to write this up!
A very perceptive observation. It is rare to see discussions connect foundational theory with practical reality so coherently.
A human using an AI has the potential to become more powerful than either a human or an AI alone.
That was true of chess engines too.
"In real world settings" I should have said.
Self-driving cars being the obvious counter-example...
IIRC LLMs already beat both GPs and GPs using LLMs in diagnostics. These will absolutely be a moment when a human is mostly in the way, and we might have already passed that moment.
(I'm a GP)
I haven’t followed the comparison studies. Have they really become this good at diagnostics?
Yes. And they're improving, unlike most practicing physicians. You can look it up if you like. An LLM can easily summarize it - I won't flood the comment section.
Am I the only one noticing the irony here?
I'm not sure what's ironic. The fact that as a practicing physician I think LLMs are better than me? Or the fact that I think LLMs improve while thinking that most physicians don't? Or me suggesting to ask an LLM to summarize data pointing to LLM superiority in diagnostics in case KN didn't want to dive too deep into it? Or me saying I won't flood the comment section with a summary I'd anyway ask an LLM to do if such summary was requested?
Are you saying the GP's knowledge and experience actually misleads them from the correct answer when using an LLM, and the LLM on its own is more likely to be correct?
That's how I interpret it, yes.
Many GPs are really just fancy autocomplete, rounding off the rich and detailed and messy information that comes from the real world into whatever they see as the most likely diagnosis from the statistics of their training distribution. They lack the intrinsic curiosity to pull apart the layers of the puzzle and achieve real understanding and insight.
Ouch.
Even so it's a little surprising that a GP using an LLM would perform worse than an LLM by itself. Wouldn't they just autocomplete whatever the LLM told them?
Should we be surprised by that? That wasn't true for basic checklists telling them to wash their hands.
The GP would do a looks-good-to-me test on the LLM output, which may have unacceptable false negative rates.
Also if the GP is working interactively with the LLM, it may trigger unrelated LLM problems that degrade its diagnostic ability (such as sycophancy towards the GP).
Probably someone good at both being a GP and working with LLMs might do better with an LLM, where the average GP with average LLM skills might not.
Yeah, I was assuming a human skilled at using AI, not current humans with an AI dumped in their lap.
If you're assuming a honed human skilled at using an AI instead of current humans, why not then compare it to a honed AI skilled at using itself instead of current AIs?
Autocomplete with a good bedside manner is probably better than autocomplete alone.
I agree! Isn't it great how consistently LLMs roleplay good bedside manner compared to some humans? Still below the top of the human range, though - they haven't saturated that benchmark yet.
Nah, that's institutional incentives. Left to themselves, most GP's would choose to spend their time talking to patients all day.
Fair enough
Most patients, yes. Some of them I can't wait to get rid of. I apologize, but that's just the way it is.
Diagnostics is not the entirety of providing healthcare.
Agreed. However, it's one of the few places where we have been able to compare humans and AI. There's a long way to go, for sure - my family's financial future depends on that - but I don't think there are many things most doctors do which can't be automated. Robotics lags decades behind, so most things done by hand (outside an operating theater) are relatively safe for now.
A psychiatrist I know routinely uses LLMs to carve treatment plans for his patients, and typically those plans are given good feedback from his seniors. For now, that process needs him to use the LLMs, but I see little reason to believe this necessarily remains the case in, say, a decade or two.
Also, one thing no one seems to be taking into account here are economic incentives. Once a large proportion of patients are consulting AI when choosing medical care, the incentives start pushing the AI's (or rather their controllers) to promote the most profitable choices.
Congrats on your son and great writing skills. U put this really well: "Can we somehow stop everything right here"
While I partly agree, I still want that life-extending nanotech without that all-life-consuming nanotech.
I don't want to stop until we have post scarcity.
We have that, it's just badly distributed
Facts. Tho if i wanted to be pedantic i could say we could use better AI's help improving wild animal welfare
And we're super wasteful, like feeding most our crops to animals and losing ~90% of the calories in the process. Not to mention that giant animal torture factories are bad in and of themself
I strongly disagree. I think that people who say things like this tend to conflate the momentary alleviation of first-world poverty with the sustainable alleviation of global scarcity. The first is counterproductive, and the latter is practically impossible.
What specific policy are you advocating for?
Distribution is a highly difficult and resource-intensive problem which we cannot solve because we do not have post-scarcity.
Post scarcity is impossible.
Post-scarcity is probably impossible except on a temporary basis without strong population control. The speed of light limit means that unfettered growth will always overcrowd the environment. (Other limits would probably come sooner, but that one's really difficult to avoid.)
The most enjoyable essay on LLM and ChatGPT I have yet to read. Dr. Reid was very well behaved, and IMO, the medical establishment is more screwed up than AI presently. I was very recently introduced to Chat as a tool for historical research. Our conversations frequently make me laugh out loud. Congratulations on your beautiful son.
6 cm and it was missed by all those ultrasounds? Disappointed radiologist here.
Why was user suspended for this comment? Seems innocuous?
I reported another comment by the same user further up the page. My rationale was that I'm pretty sure it's spam - it's not actually engaging with the comment it's responding to, it doesn't have any actual message content on its own, and there's a convenient link to the person's substack attached to the comment.
Yes, looks like you are right.
It is a bot account that has made 100+ nearly-identical comments on other blogs, with 70+ of those comments coming in a ~3 hour timespan. And yes, dumping all of those comments into a MD and having Pangram scan it shows it to be 100% AI generated:
https://www.pangram.com/history/9d15c970-c4db-49c0-880a-3b449330d960?ucc=PJF3ToqZnGm
I see. I apologize, that didn’t occur to me.
“Seems innocuous?”
It’s meaningless slop. It’s not even wrong.
(It seems like spam. It has nothing that makes it a specific reply. That is, it would have looked the same as a reply to many of the other posts here. It doesn’t relate in any real way to the post it’s replying to.)
For anyone wondering why this user was indefinitely suspended for this comment: The ban has nothing to do with the substance of the comment itself. Rather, it is a bot account:
Look at the post history, and you will see 100+ comments in the past 2 days that all follow the exact same rhetorical structure of "Brief laudatory noun phrase. A longer sentence further elaborating on the thing being praised." (They occasionally break from this pattern to self-promote.)
Within this, there is also extremely abnormal behavior that is obviously not the work of a human commenter: on October 1, the account posted 73 comments within the span of 3 hours and 18 minutes. Obviously Mr. Zambrano did not find the time within 3 hours to read 73 different posts and compose similarly-phrased comments remarking on their insight.
And yes, running his body of work through Pangram shows it to be 100% AI-generated: https://www.pangram.com/history/9d15c970-c4db-49c0-880a-3b449330d960?ucc=PJF3ToqZnGm
“The ban has nothing to do with the substance of the comment itself.”
There is no “substance”.
This one post looks like spam.
That’s what I thought when I read it. That’s comparable to the size of the uterus itself in some dimensions. How do you miss something that large on an ultrasound, let alone a bunch of different ultrasounds? I’ve got a dozen or so fibroids half that size and smaller, and they found them right away with a simple ultrasound.
That struck me as very unusual also. The hesitation to recommend an MRI seemed fair given the very low chance of it finding anything relevant that the ultrasound wouldn't have seen.
Is the chance of missing a 6cm mass on ultrasound higher than I think?
Well that depends on how likely you think it is.😁
It depends in part on patient habitus. Fat is not a friend of ultrasound. However, using transvaginal probes should lead to excellent visualization of the uterus even in the obese.
Agreed. OBGYN and I have no idea how this was missed, I really couldn’t wrap my head around it. For context an average uterus is 7 cm so a 6 cm would be like a whole extra organ… seems crazy to have gone though five transfers and no one noticed?
Nobody wants to say it because we like to think of medical professionals as tireless and perfect lifesaving angels, but like in all professions, there is a range of competence there.
Agree. And after five failed embryo transfers, the patient surely would have had a hysteroscopy where it would have been difficult to miss a large submucosal fibroid. But in any case, it should have been visible on ultrasound or sonohysterogram. Given that it was removed laparoscopically, my guess is that it was intramural. The data is mixed on whether these impact pregnancy rates in IVF. Obviously a single anecdote doesn’t prove anything. Maybe she would have gotten pregnant anyway on her next transfer without a myomectomy or maybe not. Large trials are needed to answer the question.
How are you going to get a large trial on something like that? I feel you're asking for the impossible.
She’s not really asking, just stating epistemology.
As a total noob to the topic I can only wonder: is there a chance it had only recently reached this size but had been impeding pregnancies for a long time?
I of course wonder whether it was read by a radiologist or a gynecologist.😉
also very confused by this as a sufferer of fibroids and endometriosis. If this was a story about how endometriosis was preventing successful pregnancies it would be one thing. But fibroids are not hard to spot. Even a manual pelvic exam could have found a fibroid that large, right?
Author here -- yeah, it's wild right? Here are a few more details for the interested. The problem fibroid was intramural with a submucosal component (mostly in the muscle wall), so I guess those are hard to see on ultrasound? Pathology report says 6 cm, I double checked. I'm tempted to share a pic, that thing was big and gnarly looking.
She didn’t get a hysteroscopy until after the MRI, doing that sooner would have been a good idea but it was never presented as an option.
She had other fibroids that were seen on ultrasound and removed during a surgery for endometriosis, so it's not like we didn't know fibroids could be an issue.
Obesity was not a factor, she's 5'6 120 pounds.
It might be interesting to ask your AI “why would a 6 cm fibroid be missed on multiple technically competent ultrasounds?”
Oh sure you mock now, but if GPT4 can get a young woman pregnant with one baby, then surely GPT5 can get a 50-year-old woman pregnant with twins, and GPT6 will get men pregnant with quadruplets. By GPT7, we'll all be having octuplets who themselves are already pregnant with octuplets, and the lightcone will be filled with babies by 2035.
cuteness-maximizer
That explains the catgirls.
AI safety is not "idiotic". The worst predictions may or may not come true ... but that's the case with any risk assessment. There are very reasonable and justifiable reasons to be concerned about opaque super intelligent AIs running our society. It is not "embarrassing" to be involved in a sober careful analysis of possible future risks.
One should stop making stupid anti-AI jokes or comments. Safetyism can sometimes consist in stupid anti-AI jokes or comments, but it can also consist in thoughtful discussion of the pros and cons of AI. We should try to do a lot more of the latter than the former.
"Should" isn't a very useful think to say to a population most of whom aren't listening. It doesn't work very well even to those paying attention. Actually, it doesn't work all that well even when directed at oneself...and that's really where it should be directed as "should" is a modal form of "shall".
I don't think you should conflate water hysteria, which we already know to be false with fears about growing AI capabilities.
I postulate the existence of a future god of humour who will torture you for all eternity if you don't laugh at funny anti-AI jokes today. (He will give you a pass for unfunny jokes, and if you at least crack a smile sometimes then he'll just prank you once a century.)
This is lowkey true.
I desperately, desperately hope you are right. I would relish the little thing that is the feeling of embarrassment, overwhelmed by the feeling of relief that life continues on, that all my fears were wrong, a passing thing, that children will grow up, that there's a future to look forward to.
I just don't think you are.
This was both heartwarming and depressing. Heartwarming, because of the happy ending. But also depressing, because the AI bot didn't perform any superhuman feats of AGI mental might. It just did the bare minimum that that entire health care system refused to do. People complain about losing their jobs to AI, but if you stubbornly refuse to do your job, year after year, then maybe you *should* be replaced.
Now do you understand how low the bar is for AGI? These things do not sleep, or suffer, or desire trivial things, and they can be duplicated as long as the compute necessary is provided.
They do not need to be perfect. They just need to be equivalent in terms of intelligence, and every other advantage they have will create a gap the size of the ocean between them and the people they will replace.
> Now do you understand how low the bar is for AGI?
First of all, my entire point was that we appear to be accomplishing the "equivalent in terms of intelligence" part backwards: by making humans effectively dumber. Maybe all those medical professionals involved in the story were smart in some abstract sense, but they sure didn't do anything useful until forced to do so, so their equivalent intelligence was near zero until that point. Ok, that is maybe a bit unfair; they did after all run some tests that ruled out a lot of conditions, and that is a useful result, but still.
Secondly, if a tool that can perform knowledge lookups on demand incredibly quickly, and apply them to a narrowly-defined task with some degree of reliability counts as "AGI", then sure, let's call it "AGI" -- but with the understanding that such "AGI" is still incredibly sub-par compared to the intelligence of the average motivated human; cannot operate without human guidance; and on its own poses neither an existential threat nor the promise of utopia. It merely amplifies these capabilities in its human users, much like Excel or Photoshop or BLAST, or indeed a hammer, only more so.
>> Ok, that is maybe a bit unfair
That's totally unfair. Yes I agree they should have taken the MRI earlier (I mean, anyone can tell they should have! That was the solution!), but real life medicine is hard. Most of the time - I guarantee, literally almost every time - that MRI won't do squat. And when it shows nothing, the patient usually doesn't relent, but instead doubles down and demands even more imaging studies to be done.
I know, because I'm the guy who didn't go down the path of "let's try to convince them they don't need the study done on them". I gave them everything they wanted. And most of the time, I ended up wasting everyone's time and money.
No mistake, sometimes scanning a person entirely and putting them through the full works is the solution, but please, PLEASE don't reduce this to doctors being unwilling to do their jobs. Get over here and do it yourself - without the LLM - if you think it's that easy.
For clarity, I'm not arguing against doing imaging studies in general - I love MRIs, and would love to have all studies done on myself. My point is that in real life, most of the time (especially when the first tests come up negative), pinpointing the correct diagnostic path is not easy except post hoc.
If anything, this showed both the pro and con of talking to an LLM about a medical condition. You can ask it 100 questions without it getting tired, at any hour of the day, with no irritation or shutting down the conversation. That's a major pro!
But LLM's also default to pleasing the user. Newer models can be better about this, but I still find that if I push them hard enough, they'll tell me the answer I "wanted" to hear. That's a con, particularly if your LLM is giving medical advice on a topic you're emotionally invested in.
> They do not need to be perfect. They just need to be equivalent in terms of intelligence, and every other advantage they have will create a gap the size of the ocean between them and the people they will replace.
This makes me think of self-driving cars. They don't need to be a better driver than an average human. Just better than a human whose too tired or in emotional turmoil, because the AI never will be.
> Just better than a human whose too tired or in emotional turmoil, because the AI never will be.
This is not entirely accurate. Yes, LLMs and other ML models do not suffer emotional turmoil; but they are still subject to hallucinations, mis-identifications, and other glitches. They do it all the time, in fact; but you are correct that if we could get their glitching rate below that of humans, we'd score a huge win in traffic safety.
If I read (and remember) correctly, WayMo is already claiming that. It's not clear that this would be true in all conditions, however, because they're operating in a restricted environment.
Sometimes I ask myself "what would Claude do?" and the answer is that Claude would never give up and would keep trying... and then start hallucinating... so I guess the difference is that AIs don't know when it is rational to give up, but humans often give up too soon, at least I do.
Yeah, it's definitely true that current AI's tend to go off the rails when asked to solve an impossible (under the conditions) problem, and then aren't allowed to give up.
If I work persistently for days without sleep, I will also hallucinate
I think the big benefit is making the previously scarce resource of expert attention less scarce. It's not out of malice that the medical system only gives you 30 mins with a specialist, who mostly works with a basic flowchart, but because that's an efficient use of resources at scale.
pouring -> poring
It's funny because when I got to this point I thought "Oh cool, it's human-written". (Yes I know someone could ask their LLM to include small errors, but they don't, OK?, so let's keep this trick a secret between us.)
Congratulations! What a blessing. I don't have any dramatic stories, but Dr Claude has been an absolute godsend this winter for tracking and treating all of the various lurgs and maladies that my two little kids bring home.
Can you elaborate on that? I’m curious of the benefits vs just dealing with common kid illnesses, flu, norovirus, hand foot mouth, etc
Yeah so it's stuff like:
- maintaining a detailed history of all my and my kids past health issues
- giving me the optimal treatment plan for my specific situation
- tracking when I gave what kid which medication or painkiller
- assuaging anxiety, giving me hard gates for when I should escalate to a real doctor
So none of it is revolutionary, it's more so just incredibly convenient and useful to have a highly engaged and responsive doctor-equivalent in your pocket.
To give a couple of example:
1. in the early stages of the flu, it proactively reminded me to start using a certain nasal spray that helps avoid sinus infection, which takes several days to build up effectiveness, and which I'd totally forgotten about.
2. I had some unfamiliar pain that it helped me narrow down to trapped gas, and suggested a medicine I'd never heard of before, which worked well.
Meanwhile it typically takes weeks to see a GP if it's not an emergency, and they may or may not solve the problem within my 15 minute consultation. In general there's just a lot of health stuff that never rises to the level of *needing* to visit the doctor, but where you still benefit a lot from getting good advice.
What a beautiful boy -- congratulations!
Beautiful, and congratulations to you both.
And as a resident of the Wild and Wonderful state of West Virginia, I can assure your friends that people drink dirty water not because of data centers (which are never near the bad water) but because government is incompetent and corrupt from bottom to top (and maybe because someone promised them city water in a country location, then couldn't fund the maintenance with the shrinking tax base).
He is a handsome fellow and I am glad to hear you and your wife came to a good end. Every day I find some new use for it. Lately I’ve been making my children little books. There is an upside as real as the dangers are.
This was awesome. Thanks for writing this up.
My wife and I have the same unexplained infertility diagnosis. The difficult part in my situation is that she is not as inquisitive or answer seeking as I am. I feel lost and ill-equipped to replicate your process. I don't even know if she has records of her visits and imaging.
You can do it man. Believe in yourself. Think - today is day one. You can start today
I often make this sort of exhortation to Claude before I go to bed. He gets stuff done.
If your doctor uses a patient gateway the scans and reports on them will be on the gateway.
Congratulations! And hello Reed! :D
Like everyone else, I really enjoyed Drew Houseman's write-up of his pregnancy journey. I had a related experience with Google back in 2009 when I received IVF with egg donation due to, as they called it, "advanced maternal age." I was doing fertility tourism in Turkey. My wonderful doctors were highly competent and empathetic but only explained the outline of what was being done at each visit and never gave me even a single sentence of written material. So after every clinic appointment, I attacked Google. Asking all my questions to learn what was being done to me. I suspect google sufficed for my case because mine was just a routine IVF. But with this background, it was a special pleasure to hear the saga of the parallel but more advanced research and problem-solving you received with your TV AI doctor.
I am curious about how you thought of some of the privacy concerns. I also have tons of medical questions for my doctors but I am really hesitant to let Sam and Dario know too much about my personal medical situation. In principle I guess this isn't too different from googling everything but it feels really different
I'm wondering about this - why would Sam and Dario (as in the companies they lead) give a flying duck about your health concerns? What are the negative scenarios you are thinking of?
Blackmail. Customized pricing.
I don't follow. Why would ultra rich companies alienate their user base and push them towards competition by blackmailing them? And why would they change their pricing based on the user's health data in particular?
Ultra rich companies already do this. Once your users are "locked in" you start ripping them off as hard as you can, and all sorts of personal data can help you do that more effectively. Outright blackmail is not going to happen but customized pricing definitely is. And if a company like OpenAI, Antrophic, or Google can't directly sell you things they'll sell your data to someone else who does and can profit off it, like insurers, Amazon, or even airlines.
As for why, probably because it makes them more money. But the key part is locking in your users first, if you start enshittifying before that then your users will just leave.
What sort of blackmail are we talking about? And how does the health data fit in on custom pricing LLM services?
LLMs are being practically given away for free so price gouging is not going to happen there. But Anthropic can create an accurate profile of what your needs are and then sell that to everyone else so they can price gouge you.
If you have an important appointment with a specialist that requires you fly suddenly tickets are three times more expensive. Supplements or OTC meds you've been ordering online are up in price too, it's not like you can afford not to buy them.
So, a cartel the size of the market. I suppose it's possible, but unlikely. Willing to bet this won't happen anything close to the scale you describe.
What do you mean it's possible? It's literally already happening. I genuinely wonder what world you live in if you think surveillance pricing isn't a big deal.
And yes, obviously it's done a lot by companies that have a de-facto monopoly. Amazon, Uber, Tinder. MultiPlan and Express scripts are also #2 and #3 on the Surveillance pricing index, both with high confidentiality. So yes, your personal data is used to price gouge you on healthcare. Right now. Not in theory, not in the future.
https://surveillancepricingindex.org/#companies
I'm 100% sure that we aren't living in a world where health data gathered by LLMs is used to blackmail or customize prices for LLM users in a cartel the size of the market.
I'm sorry if you interpret otherwise, but that's where this discussion started.
OpenAI and Anthropic don't sell user data yet, but Google absolutely does. Everything you say in a chat with Gemini can be sold off to a data broker and eventually used to build a detailed profile of your needs and habits for the explicit purpose of price gouging you. And selling user data is an easy way to boost profits when these companies are burning money, it's only a matter of time before they give in as well.
Customized pricing is already being used.
Thank you for the link! I've been wishing for something like that for a year or two.
I think am less worried about the AI companies themselves blackmailing me but more so the data being given to health insurance companies to deny coverage, given to employers to do medical discrimination etc.. You could image a deal where Linkedin and openAI sign a deal to do AI assessments of people's competence and interests based on chatbot messages and then the AI discriminates against people with mental illness.
I see a lot of issues with such a scenario. To begin with, a huge "delete all chats" flood for all suggested LLMs if said policies were actually threatened to be applied.
That would make me a person with a couple of dozen different cancers, several autoimmune diseases and both sexes. And a stoner and a crazy genius and a severely disabled person. How on Earth are they going to differentiate personal health chats from interests, interests from trolling, trolling from fooling around and all of the aforementioned from another person using the account?
Such behavior would also severely favor providers who promised to share less data, unless people believe said data is good for them (in which case they wouldn't mind).
"Profile hacking" would become so common sharing such data would become practically meaningless.
They also wouldn't update on future negative health outcomes as people would hide those from LLMs. So the trick only works once, and on a fixed population at that.
And so on, and on. I don't see this as a realistic future, but perhaps a will shall find a way.
What a great post. Congrats !
As a French man with a pregnant wife, I'm considering naming the baby either Claude or Astra...
I highly recommend to look up what happened to everyone called Alexa
A friend of mine, a doctor, had a case where they failed to diagnose a cancer and a young woman died because of that. Afterwards, she fed the symptoms to ChatGPT and it suggested checks that would have diagnosed the problem. Obvious stuff, like "ask about the cancer history in the family". Her point was that doctors often fail not because they are incapable, but because they are overworked and things just fall through the cracks. And that the AI could help, not because it is som kind of a super-human doctor, but because it has time that doctors don't. It can speak to the patient for hours. It can diligently check the literature. Etc. This may well have been the case here.
> not because it is som kind of a super-human doctor, but because it has time that doctors don't
Having something that humans don't seems like a decent definition of "superhuman".
Then people who are in a permanent coma are superhuman?
Clearly. Their ability to be unbothered by stuff is unmatched!
Some Buddhists spend their entire lives training to maybe get halfway there.
Patients also hate having to fill out a fourth questionnaire with the same information as the last three, minus one question about family history and plus two about MRI history, just in time to receive their fifth questionnaire (no one knows what happened to the fourth). Conditional on being fine with this sort of thing, as OP clearly is, LLMs are pure net-gain. And LLM's instant response time and good personality helps a lot. But not all people are willing to sit through that, and as others have noted, not even unlimited scans can find everything.
But the conclusion of the piece is - taking away the verbiage and shy handwringing- that ‘I got what I wanted, and I don’t know what should be done to mitigate the ghastly problems that will follow if everybody else uses A.I. to get what they want, so…there you go, eh?’ Plus baby pic, all OK with that?? What spinach.
Or... Maybe it is pointing about that all technology, from fire onwards, provides both benefits and risks and we should work to maximise the former whilst minimising the latter.
Not everything is zero sum. If we can get everyone accurate medical diagnoses faster that is just a general good.
The piece does not claim that "ghastly problems [...] will follow if everybody else uses A.I. to get what they want".
AI isn’t poisoning the water supply. The main existing downsides are aesthetics. So it’s that plus potential problems versus very real benefits.
Brilliant! Was going to write something similar. But you beat me to the punch!
This story made me very happy for you and I loved it.
Exactly related: https://www.nytimes.com/2026/09/29/health/maha-summit-kennedy-vance.html
"A.I. Is ‘Better Informed’ Than Doctors, Kennedy Tells Industry-Backed MAHA Summit"
Incidentally, this is exactly why I ask my LLM to critique my software development plans before they get implemented. It knows more than I do, makes good suggestions, and points out potential problems with my scheme, and there is no point denying it.
Just following a checklist sometimes helps a lot, but people still don't do it, because it is annoying. LLM can go through a checklist easily.
I am confused about the mental model underneath this.
The story, as told, makes it seem as if an MRI is highly unusual in this situation given that doctors do not actively suggest it and oppose it when suggested.
So what happened here?
- Over time and several conversations the AI got enough context about this particular case that it recommended a highly unusual diagnostic for this case specifically. (AI discovered something about this case that doctors missed)
- MRIs are good in any case of unexplained fertility, regardless of the particular case. (AI discovered a blind spot in the system / the system failed the author)
- MRIs are something doctors in medical TV do a lot so if you ask an AI to write a script for long enough at some point it will recommend an MRI, no matter what the diagnosis, let alone the particular case is. (AI got lucky)
- Something else.
Great question!
I could give you a "Bayesian" argument that after you tried the usual things and they failed, you need to consider the unusual things (proportionally to their prior probabilities and the evidence you have received so far), which could push MRI from "unlikely to be useful" to "likely to be useful".
(Something like -- numbers completely made up -- there is only a 1 in 1000 chance that MRI would help a random person. But we already tried things that would have helped 995 out of 1000 people, so now you are one of the remaining 5, and there is a 20% chance you are the one who would benefit from the MRI.)
But why the LLM actually did it, that is a completely different question.
Thank you for mentioning this. I'm optimistic about AI in medical diagnostics, but Drew's methodology seems very unsound.
For one thing, asking an AI to roleplay, especially using an explicit reference to popular fiction, seems likely to elicit character-driven behavior, with all of its flaws, rather than true rationality. I have a similar critique of the use of AI for personal coaching.*
For another, some of the questions that Drew asks the AI are presumptive and leading. Asking "is there a problem?" is a lot different from asking "what's the best evidence for a problem?", and I think that the question in the third screenshot ("[W]hat might [you] see that's concerning?" seems closer to the latter.
*https://matthewmilone.substack.com/p/companions-are-not-coaches
There’s a saying in medicine that even a blind pig finds an acorn now and then. I’m surprised an AI prompted with “Dr House” didn’t suggest an MRI much earlier, Dr House orders them like DoorDash burritos.
The MRI episode is a useful example of why fluent medical assistance needs a verification boundary: the model surfaced a diagnostic path, but the clinician still had to validate it and act. The strongest workflow seems less like replacing expertise and more like giving patients a persistent, context-aware question generator whose suggestions are explicitly checked against records and guidelines.
Congratulations on the baby!
Oh wow this is so cute!
Pre-AI I had my own experience with infertility which eventually resolved in first a +ve on a pee stick and then a baby with all the normal number of limbs and digits but it was a weird time.
Great tale! They should make a movie out of it (at least an AI-generated short film) where the actual events are portrayed realistically, but the "Dr. Reid" interactions are depicted as a cheesy TV drama, with stilted writing (but very good and specific medical advice!) and obvious AI artifacts all over the place. And Dr. Reid also gets a happy ending where the happy parents show him the baby named after him.
This could be an entertaining and more lighthearted than usual episode of Black Mirror.
I'm happy for the author and his wife. Still, I always feel ambivalent about this kind of story. The natural conclusion seems very favorable to AI and very unfavorable to human doctors and the medical establishment.
But the same story could have been told twenty years ago with no AI in the loop. Desperate patients determined not to give up would have kept seeking information from other healthcare professionals, patient associations, and so on. They would have read and reread the medical file and stumbled upon the document suggesting an MRI, or someone else would have suggested it. They would have pressed the doctor in charge of the case until they got it, and the outcome would have been the same. It's worth keeping in mind that medical errors, or plain negligence, have always existed.
Even setting fault aside, it's also a matter of how much one invests. How much time, energy and money is it reasonable to commit to a goal whose outcome is uncertain? To take a reductio ad absurdum: a patient who exhausts all their time and resources trying to get to the bottom of their infertility, to the point of losing their job and ending up on the street. If they fail, we might be tempted to call them unreasonable. If, against all odds, their efforts paid off and they later got back on their feet and lived a good life with their children, we might admire them instead. We are quick to judge a situation by its outcome, and the author's story is no exception. He probably wouldn't have published it had it ended in failure. There is inevitably some bias here, since this is an individual story with no counterfactual.
In short, it's hard to conclude anything from a single anecdote, let alone get excited about AI assistance on that basis.
That said, this view coexists in my mind with a less guarded one. There's no denying that technology brings a great deal to medicine. In particular, the very existence of MRI may matter even more than AI in this story. It would be naive to dismiss what technology contributes. The surgical robot is an asset, and so is AI.
At the time of the events the author describes, I again don't think AI was entirely a game changer, since he could probably have reached the same outcome by combing through the medical file again and again and continuing to consult various people. The result is mainly due to the couple's commitment and determination. Back then, GPT-4 was not unambiguously superhuman in medicine.
However, the trend lines keep going, and yes, there will come a point where AI is vastly superhuman in medicine. I wouldn't be surprised if that were already the case. What's at stake is whether this technology will empower doctors, as MRI and surgical robots have, rather than disempowering them. That remains an open question. There is also the concern that this empowerment period could be very short before desempowerment or a tragic loss of control.
They didn't need to exhaust all their time & resources, because the AI made it easier for them to talk to a "doctor" whenever they wanted.
I don’t know if you meant this, but your “what’s at stake” paragraph suggests that the important thing is the consequences to doctors, rather than the benefits to patients.
So nice outcome. I'm very happy for the author and his wife.
Ehh.
I'm ill for 11 years, in a terrible neurological pain all over my body. (Allodynia?)
Nobody knows why. Unfortunately, it does not look that my issues can be solved by a measly MRI. (I did several).
I tried the same yesterday, asking an AI chatbot. Unfortunately, most things it outlines I already did try.
Usual pain relievers, SSRI, tricyclics, gabanoids, massage, exercise, avoiding stress, not working, sleeping a lot, yoga, swimming.
The only things that helps is opiates, and this is the road in one direction. I'm going there, and death is closer with each step.
Maybe I need to do more tests for nerve damage (but how to treat it anyway?)
Your best hope is to make yourself as expert as a layman can be on your problem. I recommend you get on one of the really good chatbots (I use GPT Astra) and have it do a thorough "consult," aiming first to identify possible causes of your pain. Give it an initial brief summary of the problem then ask it to interview you in depth and ask for any records it needs to see, aiming at this point to figure out what the possible diagnoses are, not what treatments might work. Upload any records it asks for. Ask it to make a list of the likeliest causes of your pain, and for each to name the features of your case that make it a good fit, and the ones that don't, and what tests are needed to determine whether you have that diagnosis, and what treatments there are for each, and how effective. And it is worthwhile, for each possible diagnosis, to ask whether there are any novel, effective treatments for it or any old effective ones that are currently overlooked. Get the AI to summarize the research on diagnosis, how illness works, and treatment, asking that it use only reputable research sources.
This process will take several hours. It will be time well spent.
Then take it from there. You should probably see a doctor at that point, using the AI to find doctors with special expertise in the problem you suspect you have.
Very heartwarming <3
Wow, this was great, thx. (I can feel a small tear starting in the corner of my eye... :)
Congratulations on your baby!
Our life is all about our son right now and it's hard to think about how close a call it was for him to actually come to being. Our doctors were really good all along the way, which has been part of it -- they were super knowledgeable and gave us plenty of time at each visit. Even then, a lot of things had to go right for us to have our healthy baby (in particular some well-timed scans that discovered a severe fetal growth restriction).
Congratulations! Heart-warming story
Beautiful story. Thanks
The part that got me is the AI didn't out-diagnose anyone. It just read everything, which is the one thing nobody in the system has time to do. Six years, one fibroid, zero fresh eyes until a $20/month chatbot. That's either an indictment or a business model, and I'm not sure which... either way kinda scary
Thanks for the article!
It’s unfortunate you have to use jailbreaks / workarounds to get an answer out of ChatGPT.
Deepseek will usually answer medical question for me without needing jailbreaks, although it does sometimes get scared looking at my data. (To be fair on DeepSeek here, both my GP and my endocrinologist are kind of twitchy about my test results, for reasons where DeepSeek is thinking ahead of them)
You could have bootstrapped Reed to ASI with the Dr. House persona if you told it to continuously pretend to take Vicodin and spin up other LLMs there to act exasperated at him and talk about how all the tests have failed and surely there's no solution while making meta comments that they are approaching the last 10 minutes of the episode.
Congrats! Similar if less dramatic story, my wife and I have been using Claude to generate social stories for our 4 y/o who can have a lot of difficulty navigating new situations and encounters flexibly. We upload a few pictures and a prompt and it starts chugging away. It's part of an ongoing project so it can also reference previous social stories and all our other notes on him so it's very personalized. Our son loves them and I think it's also helpful.
So yeah, thanks robots, that's good, please don't get murderous.
Thank you! And haha yep, no murders just helpful stories please. That’s a really cool use case
This is absolutely insane! Thank you for sharing & congrats on your beautiful baby!
Congratulations! This reminds me of a story about someone who consulted with and LLM in (what turned out to be) a medical emergency:
https://benorenstein.substack.com/p/chatgpt-sent-me-to-the-er
Unrelatedly, I've struggled to convince EAs and rationalists to think of prenatal humans as people, so I was encouraged to see you casually refer to your frozen embryos as "babies".
Challenging for sure! It would need to be multi-center, would probably take years to enroll enough subjects, and would cost a lot of money.
Congrats on your cute son! Nice write up. To me, this seems like a story about medical incompetence than anything else.
As others have said, this was not a particularly sophisticated diagnostic suggestion. I definitely support using AI to double check your physicians plan. Although have certainly seen a couple go off the deep end asking for research-level testing for mostly theoretical conditions. I’m surprised by how few of my patients seem to use it so far.
I has a similar experience with ChatGPT and IVF!
I did my first round just doing exactly what the doctor told me. We got 8 euploids, which is not bad, but I felt we could do better. Next round I let Dr. Chat take the wheel, and asked my human doctor to follow Dr. Chat's recommendations. We got 14 euploids! And for round 3, 16 euploids!
I think we are officially at the point where the shoggoths are just better than the average doctor (not surprising since real world doctors have time constraints whereas Dr. Chat does not).
I always run my medical stuff through Dr. Chat now, even for minor stuff
(Big changes it suggested were longer stim period with a lower amount of stimulation hormone, and conventional fertilisation rather than ICSI)
That’s so awesome! Good luck on the next steps of the journey
There’s a kid out there named Opus, probably.
MRIs are one of those technologies that is great for individuals but not for populations, necessarily — not everybody needs an MRI but Drew’s wife did, and does my friend whose cancer was detected after they insisted on getting an MRI — but there’s tons of useless MRIs too.