The Aboard Podcast
Kamal Menghrajani: The Limits of AI Healthcare
Doctors might be using AI to cut down on paperwork, but can these tools really be employed in clinical settings? On this week’s podcast, Paul and Rich sit down in the studio with Dr. Kamal Menghrajani, a practicing oncologist and lecturer at Harvard Medical School who was previously a member of the Biden White House’s “Cancer Moonshot” team. After she explains how AI is helping in her work, she lays out its real limitations—and discusses how these technologies can distract from more systemic approaches like better patient prevention and screening. Plus: Are her medical students allowed to use LLMs?
Show Notes
- Kamal’s website
- The Cancer Moonshot initiative
- “Estimation of Cancer Deaths Averted From Prevention, Screening, and Treatment Efforts, 1975-2020”
- Andrew Leland on the podcast
Transcript
Paul Ford: Hi, I’m Paul Ford.
Rich Ziade: And I’m Rich Ziade.
Paul: And this is The Aboard Podcast, the podcast about how AI is changing the world of software and the world in general. Hello, Richard.
Rich: Hello, Paul.
Paul: I want us to play the theme song, and then I want us to talk to our guest, because this is a really good one.
Rich: Let’s do it.
[intro music]
Paul: For just a few seconds, explain what Aboard is, because people keep asking.
Rich: Let’s do it.
Paul: You want to give me, like, a one sentence?
Rich: One sentence??
Paul: That’s all you get.
Rich: Give me three.
Paul: Okay.
Rich: All right. We ship amazing AI-powered solutions for companies. We go in, see what you need, see where there are opportunities to run your business better or your organization better. And then we deploy amazing people and amazing technology to get you there.
Paul: That’s pretty nice. We have about 36,000 years of combined experience building tools, and we’ve been riding this wave and figuring out how to actually drive real value and actually ship stuff.
Rich: Quite the wave. It’s quite the wave.
Paul: All that stuff you hear about 95% of AI projects not shipping? That’s because 95% of people could try harder. We can do it for you.
Rich: The other 5%? That’s us.
Paul: That’s us! 5%. So good example, go to our website, check out Make An Impact, the case study. It’s, you know, we built a big medical dashboard that you can talk to, you can ask it questions. Making data actionable, making things work, keeping things under the rules and regulated and compliant. That’s enough about us.
Rich: Enough about us.
Paul: Let’s go to our guest. Welcome, Dr. Kamal Menghrajani.
Kamal Menghrajani: Hey, thank you guys so much for having me.
Paul: So I have to tell you, we’ve looked at your LinkedIn and there’s a lot going on.
Kamal: Yeah.
Paul: And you know, doctors are funny because they do a lot of different things. So help us understand which of the different things you’re doing.
Kamal: Yeah, I think right now I am a practicing oncologist who is working deeply to think about how we can improve healthcare using AI.
Paul: Okay, you’re teaching. You’re…
Kamal: Yeah.
Paul: Well, I mean, give me some more. Give me a little more.
Kamal: Sure. So how did I get here? So I am a classically trained physician scientist. So I was actually at Memorial Sloan Kettering for eight years, working on genomics using computational oncology approaches to better understand how do we understand cancer. Who’s going to get it? What’s going to happen when they do? Are they going to respond to treatment? And it was great sort of writing software from scratch and trying to understand these big, thorny questions and get answers.
Paul: Oh, so you’re programming, too.
Kamal: Yes. So that’s my back—that’s the link. That’s part of how I got into this.
Paul: Okay.
Kamal: But to learn to program, I actually went to school at Columbia, at the School of Public Health.
Paul: Mmm hmm.
Kamal: And I got a master’s in statistics. And they said, “Yeah, we’ll teach you statistics, but you have to learn all of public health with it.” And that got me really curious about, okay, what are the other intersections of public health and oncology? And so from there, that interest led me to become the oncologist for the Cancer Moonshot.
Paul: Mmm hmm.
Kamal: So I moved to Washington, D.C. I worked at the White House under the Biden-Harris administration for a year and a half. Focused both on the Cancer Moonshot, but also became the physician for the health team.
Paul: Okay.
Kamal: So I ran point for federal guidance and legislation that was coming out from FDA, CDC, CMS, and really had this 30,000-foot view of what health policy looked like in this country.
Paul: So simultaneously on the ground, lots of patients, lots of stuff going on, and then also way, way up.
Kamal: Yeah.
Paul: And sort of looking at cancer at a global and national scale.
Kamal: Absolutely.
Rich: With an incredibly ambitious mandate.
Kamal: Yeah, the Cancer Moonshot, I mean, had these two big goals. First of, how do we decrease the cancer death rate by 50%? How do we cut it in half over the next 25 years, essentially from when it was launched? But then thinking about the softer side of, how do we improve the experience for people who are facing a diagnosis, whether as a patient, a family member, or a caregiver?
Rich: Mmm hmm.
Kamal: So really thinking holistically. And even though a lot of our work was focused domestically, we ended up launching a global effort, which a lot of us are still continuing now that we’re back as private citizens.
Rich: Before we get into what you’re doing today, tell us what you learned, I mean, obviously the mandate is hovering over everyone. You’re trying to get this done, but obviously there’s a lot of people involved, a lot of process, a lot of stuff. Right? And we’re gonna come back to this. The culture and the people that are in the mix here. What’d you learn? What’d you learn with this—okay, here’s this incredibly ambitious mandate. They parachute you into D.C. [laughter] She’s like, “God, you asked me that one?”
Kamal: Such a big question. It’s interesting, I think there are incredibly smart, well-intentioned people who are, are working in D.C., at least during my time, you know, during the Biden-Harris administration, who are really thinking about, how do we improve health care for as many people in the American public as possible, and how do we do it in a way that’s guided by evidence?
Rich: Mmm hmm.
Kamal: And so, you know, coming in as somebody with expertise in clinical practice and using evidence to make decisions, it was really wonderful to say, okay, I’m used to doing this on a patient-to-patient level. How do we think about this on a systems level? How do we think about this on a national level?
Rich: Mmm hmm.
Kamal: That was one of the biggest takeaways, and I think the other was a lot of these decisions are made by very small groups of people. You know, when you sit in the room where it happens, you’re looking at who else is there at the table. And it’s really amazing, you know, thinking about who is there and what viewpoints are they representing.
Rich: Mmm.
Kamal: And so now that I’m outside of government, thinking about, okay, well, what are the things that we can do to get things moving, to start to advocate, to start to use private industry to move forward? Some of these same public health principles.
Rich: Mmm hmm.
Kamal: And how do we make sure we make a big enough splash so that the people who are sitting around that table have a sense of where we think healthcare should be going?
Rich: Right.
Kamal: And I think that’s why working in AI right now is so exciting.
Rich: Right. You’re inside this big initiative and then AI is, you can see it in the distance. What was the timing of… Obviously there’s a lot of conversation, well, we can finally find some cures. AI is here, right? [laughter] There’s a lot of that sentiment.
Kamal: AI is gonna fix everything!
Paul: They’re big on that, too. They love to say, I mean, cancer would be, you know, it’s just a product feature, though. They can cure it at this point. [laughter] That’s Sam Altman out there…
Kamal: I mean, just get a few Claude Code agents going, you know, we’ll solve the whole thing.
Paul: That’s right. We just need to get you another Mac.
Kamal: I mean, honestly, just need a little bit more—get me a few GPUs and we’ll just, we’ll knock this out.
Paul: Problem solved.
Rich: I mean, look, there’s a lot of people outside looking in who are very hopeful about this convergence of technology and what we’re trying to solve. But based on your mocking laughter, it’s not that simple.
Kamal: It’s not that simple. The part of the White House I worked in was called the Office of Science and Technology Policy.
Rich: Mmm hmm.
Kamal: And literally it was us on the health team, and next door, on the same hallway, was the tech team who were developing the AI strategy. So these two things were actually very intertwined. And one of the big initiatives that came out of the Cancer Moonshot was a project called Cancer X. And they have multiple functions, but one of them is to serve as an accelerator program for AI specifically in oncology. They helped some nascent startups get up off the ground, helped them create a minimal viable product so that they could actually see what are the different ways we can apply AI to solving this big thorny problem around cancer.
Rich: Mmm hmm. Right.
Kamal: So the use of AI, even three, four years ago was something that people were thinking about. Of course now the technology has evolved to the point that we’re able to do much more than we were back then.
Rich: Yeah.
Kamal: But this is a train that’s been chugging along for a while now.
Rich: All right, so you’re part of this big initiative. You’re in D.C. I guess you paused your practice, I’m assuming.
Kamal: Yeah, absolutely.
Rich: To go do this civic duty. Let’s go do this big job.
Paul: Did you move there?
Kamal: I did.
Paul: Oh, wow. All the way. All the way in.
Kamal: Yeah. My husband is also an oncologist and he works here in New York. And so I moved to D.C. and he stayed here in New York, kept his practice going. He’s a professor in a medical school and so he kept teaching. And so when the Biden-Harris administration came to a conclusion, I just moved home, and so came back, came back to New York and decided, you know, I wanted to take my career in a different direction. Having seen the 30,000-foot view, I realized, you know, I thought AI was really going to be the next big thing in terms of thinking about how do we affect change to the healthcare system overall? And so wanted to go back into clinical practice and now am up at Mass General Hospital and MGH, affiliated with Harvard Medical School and practicing as an oncologist there part-time.
Rich: So you are practicing?
Kamal: I am practicing and then working in AI consulting, you know, working with different folks who are creating AI tools, which has been really exciting.
Paul: I love—you know, typically in the tech industry, when somebody’s like, boy, you know, kind of got a little out of control or got a little big and I need to wind down, they open like a bakery. [laughter]
Rich: Yeah, yeah, yeah. Food truck.
Paul: They’re just, or like, no, no—or leather working, that’s another one. Or carpentry. Anyway, okay, so, so here we are. And so you’ve had this, this view into public health that very few people get to see where you’re kind of across the whole country across the whole world. And this alien object lands, and everyone starts telling you that their foot hurts, and then they tell their doctor what ChatGPT says should happen.
Rich: Printouts.
Kamal: Yes.
Paul: And then you leave that world. Like, you come down from 30,000 feet where you’ve been hovering, and now you’re sort of back on the ground. Like, I don’t know. Just contextualize that. What is AI to you as a doctor? What was it in terms of public policy? How is the medical community reacting to the fact that everybody would rather tell ChatGPT what’s wrong with them than a medical professional?
Kamal: Yeah, I think there’s so many facets to this. Just as AI is affecting change in so many different sectors. From a consumer perspective, I think it’s a good thing if patients are more educated. I think it’s a good thing if patients are more engaged with their health. It’s not new as a clinician for a patient to come in and say, “Hey, you know, were you aware of this study? Did you know what this research was saying?” Patients have been doing that for a long time.
Paul: You can snort shampoo and it’ll fix your glaucoma! [laughter]
Rich: Yeah, it’s the WebMD printout.
Kamal: Yes.
Rich: But the first two pages are ads. [laughter]
Kamal: Yes.
Paul: Oh, it’s bad. It’s bad.
Kamal: I mean, people, you know, people come into me as an oncologist and they say, “Hey, you know, a friend of a friend gave me this New England Journal article. Have you read it?”
Paul: Okay.
Kamal: And, you know, that has been happening for a long time.
Rich: And you don’t want to be dismissive, I’m guessing. A patient’s in a vulnerable place.
Kamal: No. Yeah.
Rich: Yeah.
Kamal: And I think they’re trying to learn and get up to speed.
Rich: Yeah.
Kamal: On something that requires, you know, decades to train and really understand about.
Rich: Sure.
Kamal: So…
Paul: This is a really interesting thing, because people think that information will yield knowledge very quickly, as opposed to needing to bake for decades.
Kamal: Yeah.
Paul: Which is something we run into a lot. Like, AI can write a lot of code for you. Right? It’s just amazing. Thinking about a system and how a system interacts and what’s going to be compliant and all that stuff just is really thorny and ugly and sort of complicated. And so I’m not surprised. Right? Like, I found this PDF, and it might… Everybody who finds a PDF thinks they might have unlocked the secret to the universe. Right? [laughter]
Rich: In a PDF.
Paul: And here you are, and you’re like, it’s much worse than you could ever imagine.
Kamal: It’s not necessarily worse, but it is…
Paul: It is intact.
Kamal: It is more nuanced, for sure.
Rich: It’s never that simple.
Kamal: It’s never that simple.
Rich: Almost ever.
Kamal: Yeah. But, you know, it does provide an opportunity for the physician to then respond and say, “Great, I’m excited, excited that you’re interested in learning more about this. Let us, you know, let’s set some context. Let’s talk about it from the beginning. Let’s zoom out, and then let’s talk about where what you found fits into the overall picture of what we’re doing.” And so it’s a way in, to have that more engaged conversation with a patient. And I think that’s always a good thing.
Rich: Yeah.
Paul: You’re all so nice and patient. [laughter]
Rich: Well, no, I think. I’m sure the opposite, I’m sure some physicians get defensive and they’re kind of annoyed about it. And it’s like, “What are you doing?” And I think your tone and your approach is really wise, Especially now, like the cat’s, there’s this talking robot for everybody, right?
Kamal: [laughing] Right.
Rich: Tell us, let’s shift into, I mean, AI is a chameleon. It can do a lot of different things. Tell us how you think about it delivering value in so many contexts and use cases that, you know, a lot of people don’t know about. Right? We’ve had events here, and I was surprised that it’s made its way in. I thought the medical community was going to be, “You stay, wait at the door. Not ready for you just yet.”
Paul: First, I’ll just offer some brief context, we did this event. You spoke, it was great. And what Rich and I learned was that doctors, many of them love AI.
Rich: They’re in.
Kamal: Yeah.
Paul: You work with a company called Doximity, and, like, it just published a survey of many, many doctors. And, like, they’re in. They’re like, help me transcribe and so on. And I thought about this for a long time because I’m very adjacent to a lot of creative industries. They hate it. Technologists are like, all right, it’s going to replace programming. Something was going to do that eventually, it’s okay. But then to hear the doctors be like, “I don’t have to stay up late.” [laughter] Doctors are so excited.
My thesis was that the industry is so regulated and there’s so many guarantees that things will be kind of structured and the rules were already there, that they don’t feel that the rules are under siege in the same way that other industries are. Like, if somebody is like an illustrator and ChatGPT can draw, they’re like, “Well, that’s it. I don’t, there’s no defense. I don’t have, there’s no AMA. There’s no—”
Rich: They’re human.
Paul: Yeah.
Kamal: Yeah
Rich: They’re human. My doctor, very well-regarded neurologist, didn’t respond to one of my messages. And then he emailed, he finally responded, and his first sentence was, “It’s been nuts.” [laughter] And I was just like, “Whoa, Doc! You’re like the vice chair of the department.”
Paul: No. It is funny, as you get older—
Rich: But it was very, I actually appreciated it. It was like him, the guard came down, it was like 10 o’clock at night, and it’s just like, “I’m sorry I didn’t get back to you. It’s been nuts.” And I was like, “Okay, I get it.”
Paul: No, there is a point, there’s like, I’ve seen my GP for like 15, 20 years, and there’s a point where they’re just like, then they like to talk. Right?
Kamal: Yeah.
Rich: It takes time.
Paul: The formality slips through. And then, and then they’re like, “I got the one son. He’s driving me insane.” [laughter]
Rich: So walk, we just said, we just made a blanket statement. Doctors are using it. Walk us through how they’re using it and what you’re seeing.
Kamal: Yeah.
Rich: What’s your reaction to that?
Kamal: You know, as a starting place, I think there’s a lot of places in society where people are really worried that AI is going to take their jobs. So, you know, paralegals is one example, you know, where it’s—
Rich: Yeah.
Kamal: Here’s this profession that now, you know, AI can, can do so much faster and reasonably accurately that, you know, you can understand that those people might be a little bit scared that their job may be obsolete in the age of AI. And I don’t think doctors necessarily see that same existential threat.
Rich: They don’t?
Kamal: Yeah.
Rich: Yeah.
Kamal: And I think that’s a part of why they’re so excited to use these AI tools.
Rich: Interesting, yeah.
Kamal: Most physicians, if you ask them how they like spending their time, most of them like spending their time on their craft. So whether they’re a surgeon, they want to be in the OR. I’m an internal medicine-based specialty. You know, I do oncology. I really love spending time with my patients. I really love teaching my residents. That’s where I want to be spending my time. And so if—
Paul: And where were you spending your time—
Rich: Pre-AI?
Paul: Yeah.
Kamal: Yeah. I think a lot of time is spent not necessarily just by the physician, but also by the staff on things like writing documentation. So…
Rich: Paperwork.
Kamal: Paperwork. Someone comes into my office, I have to make sure that’s documented because that’s how I submit a bill and get paid. And some doctors are spending almost twice as long documenting as they are with the patient. And a lot of doctors go home at the end of the night and they have to finish their patient charts. So this is something that is called pajama time, where, you know, you’re at home, you’re finally comfortable, you’ve taken care of your kids or whatever you have to do, and instead of relaxing or enjoying time with your spouse or doing a hobby.
Rich: You open your laptop.
Kamal: You’re back online, you’re back on your EHR.
Rich: Yeah.
Kamal: Trying to finish up the work.
Paul: It sounds so fun. [laughter] It’s just like, “Hey, guys, pajama time!”
Kamal: Pajama party!
Paul: Yeah!
Rich: Yeah, you mentioned it in passing, but to get paid.
Kamal: Yeah.
Rich: It’s not just. I want to have thorough documentation so when this patient comes back and visits, I have full context.
Kamal: Yeah.
Rich: It’s also just the grind of the bureaucracy of getting paid and all that.
Kamal: Yeah. There’s some overlap there between, you know, I want this note to reflect my decision-making and my thinking so that when I come back to it, or if a colleague, you know, is going to be consulting on this patient, I want it to be clear what it is we’re doing and why. But sometimes there’s parts of that note that you don’t need to keep documenting every time. They’re not directly relevant to what you’re thinking, but you have to make sure you put it in there, because that way, when you submit a bill to the insurance company, you can make sure your time is reimbursed appropriately. And so if AI can help you complete that documentation, that is just such a huge win.
Paul: This is the other thing I notice when I go to the doctor is it doesn’t matter what, like, I broke my foot and I’ll tell them I work in software, and they’re like, “Boy, maybe you could fix this.” And then they’ll point to the health-record system.
Kamal: Yes.
Paul: So it’s just, that’s a grind. Nobody seems to like their tools. Nobody seems to like their sort of digital existence as a doctor, because it’s just taking them away from all the other things they care about.
Kamal: Yeah.
Paul: And it seems pretty universal, as far as I can tell.
Kamal: Yeah. And, you know, sometimes if you’re new to a system or new to an EHR or, you know, just the way it’s set up, getting the information out of the record that’s relevant so that you can make decisions is sometimes painstaking.
Rich: You’re squinting and scrolling through all sorts of stuff.
Kamal: You’re reading through note after note, trying to figure out which one might actually have the relevant information. And it can really squander a lot of time. Whereas if there were an easier way to just get that information out of the EHR, you could then be like, okay, I’m going to go into action because now I have an understanding of this patient and their history and the rich background, everything they’ve been through. I have the context that I need to make the right decision for them without having to spend hours just sorting through this information, which is not well-structured.
Paul: You know what’s fascinating here, what I’m pulling out, is that so you’ve got the AI companies and they’re like, “Oh, hey, Cancer Moonshot. That was cool. But wait till you see what we can do.” Right? [laughter] And so it’s just, everything is this incredible aspirational future state where everything will be solved and the human body will no longer be a mystery and et cetera, et cetera.
And then we’re hearing from doctors and doctors are saying, “I have way too much paperwork. I need, just, could we start with clerical first?” And I always think of this as like, the right way to bring AI into the organization is to clean up the mess the old computers made.
Kamal: Yes.
Paul: We’ll take the new computer, we’ll clean up the old computer’s mess. And I think, you know, our audience, it’s a lot of product managers, a lot of people work in tech. Healthcare feels almost insurmountable if you’re not in it.
Kamal: Mmm hmm.
Paul: Because it’s also when you start to work in it, people are like, they get very sort of mythological about HIPAA. A lot of this stuff is very easy to solve, but it’s just as an industry, it’s got a very funny relationship with tech. But most of the challenges, so many of them are literally, like, I just need to find documents more readily, I need to transcribe things more efficiently. Like, it’s so many simple things that you’re describing that are not related to, like, cells. They’re not related to, you know, internalizing and dealing with huge data sets. They’re just like, can you just make it so I can go to bed?
Kamal: Yeah.
Rich: I mean, I guess I’ll pose it as a question. Adoption via, “Oh, I’m going to email hospital IT right now that I found a really cool piece of software that could make me more efficient,” and then it just goes into a void and never comes out.
Kamal: Yeah.
Rich: And then there’s, you know, what we call grassroots adoption, which is like, “I can get an app on my phone that’s going to listen to me talk and then I’m going to copy-paste some stuff and I route it around the organization, to a large extent.” I mean, how did, I’d be shocked if you tell me this adoption is the result of a lot of large IT purchases from hospitals.
Kamal: [laughing] I think there are… So, you know, it’s interesting when you think about where is AI seeing the most utility in healthcare today? So one is what we were talking about in terms of can you speed up the documentation?
Rich: Yeah.
Kamal: One way that AI is doing that you’ve referred to is sort of this ambient dictation software. I have a clinic visit, somebody comes in to see me, I bring in my phone, I turn on the app, it’s listening to the whole conversation, and it turns it into a note at the end.
Rich: Yeah.
Kamal: Now I don’t have to do that typing. And so that can be really useful and a real time saver. The other place where there’s a lot of AI adoption is what’s called clinical decision support. So I’m taking care of a patient. You know, I do oncology, but this person, there’s something going on with their kidneys. I’m not entirely sure. And so I’m going to put some stuff into this AI and it’s going to tell me, “Oh, you should be thinking about these five potential problems. Here are the next steps to work it up. Why don’t you order these labs, order these tests, and then you’ll be able to figure out what’s going on with their kidneys without necessarily having to go and call a specialist right away.” And it gives, these clinical decision support tools will also give you evidence. So they’ll give you, here’s the literature, here’s the New England journal paper, here’s the guideline that is supporting this systematic way of trying to work up what’s going on with the patient.
Rich: Wow, that’s a big deal. That’s not just note taking.
Kamal: It’s not just note taking. It’s really helping physicians think through problems in a way that’s evidence-based and up to date.
Paul: I think there’s an important thing. I’ve noticed this, like, I didn’t realize this about the craft until we had twins. It was a high-risk pregnancy and I did very little. And, but I didn’t, as we got further and further in the pregnancy, I realized there was a weekly meeting about our pregnancy along with all the other things going on. Right? And that—doctors talk. And I don’t think when you are a patient, you realize that, because when you’re saying this, like, that information comes in, but it doesn’t just stay with you. Like, you’re going to talk about what, you’re not just going to be, like, okay, the AI said it, it’s good. Which is kind of what’s happening elsewhere in the world. You’re going to take this information kind of into the culture that you’re in, you’re going to share it out, you’re going to be like, I think this, I think that, and you kind of don’t…like it just always struck me because we think of doctors, I think a lot of times as just kind of these brains [laughter] that operate—
Rich: Input, output.
Kamal: Brain in a jar somewhere.
Paul: Yeah!
Kamal: Yeah. Floating around.
Paul: We don’t think of you like with a PowerPoint presentation in a room somewhere in the hospital with four other doctors—
Kamal: Right.
Paul: Being like, “Man, I don’t know. What do you think? Is this…?” [laughter] The idea that you could all be somewhere shrugging is alien to patients. Right? But I’m sure it has to happen all the time.
Kamal: Yeah.
Paul: Yeah.
Kamal: People don’t read textbooks. They present the way they present. And the study in the New England Journal or whatever other, wherever you’re finding it, it’s done in a specific population in a very controlled way. And most of the patients who present don’t 100% fit. And so that leaves a lot of white space in the practice of medicine.
Rich: Sure.
Kamal: And so there’s a lot of room—
Rich: Judgment.
Kamal: Exactly. A lot of room for experience. You know, people who have tried something again and again and again and have seen it work. And so we do our best to practice evidence-based medicine. And I think AI can be really helpful in terms of, here’s what the evidence shows, here’s the, you know, the ideal way of facing this issue. But there also needs to be more space where clinicians can discuss difficult cases, cases that don’t fit the regular paradigm, and bring in that clinical expertise to layer on top. And I think AI is not there yet.
Rich: Yeah.
Paul: No. I don’t know if it can be. What do you think?
Kamal: I think it’s a great question. I think this could be a real differentiator for where AI is headed. I think if we can find a way to not only integrate, here’s the literature, here’s the guidelines, but here’s clinical expertise on top of it, allow for more of those physician to physician conversations to happen, not just in the same room with a PowerPoint, but going on across the country or potentially across the world, then I think we’ll really be able to take these tools and turn it into improved health outcomes in a way that wasn’t possible before.
Paul: There’s another thing I’m thinking a lot about, which is LLMs as a technology, it’s very hard, they’re not inherently reliable. They’re trying to make them reliable, but they’re not inherently reliable. Meanwhile, the history of medicine has an unbelievable number of things around knowledge graphs and sort of data-driven methods and machine learning that are far more reliable. And I think we’re just at the very beginning of making a loop between those things. Like, the LLM is great for querying. It can figure out what you’re asking, you can get a lot of intent and then you can go consult really large corporate databases like vast sets of PDFs, but also vast sets of data. And I think that loop is going to be really exciting, where you’re like, “Hey, help me get into this database of prior outcomes and figure out what happened over the last 20 years.” And I think it can be an amazing interface for that. But I think that world is just starting, where we sort of bring those two together.
Kamal: Yeah, I think there are some enterprise AI solutions for healthcare that I imagine are trying to do this.
Paul: Mmm hmm.
Kamal: And it brings up a few interesting questions. Number one, how much of the information that is actually used for patient decision-making is structured in a way that an LLM has access to it? Because yes, you do get the labs and you do get the radiology read, so on and so forth, but so much happens in conversation, and that’s not captured anywhere. So is the data that’s even available to the AI adequate to make appropriate decisions?
Rich: Is that data even making its way back to the LLMs, though? Even the structured stuff? Probably not.
Paul: No.
Kamal: Structured notes are sort of a different thing. There are efforts in different EHRs to try to structure the data better.
Rich: Anonymize it, get it out there so we can see patterns and such?
Kamal: So when we think about how do we actually use patient data for research, which is sort of what you’re alluding to?
Rich: Yeah.
Kamal: How do we find patterns?
Rich: Yeah.
Kamal: There’s a few things involved. Number one, it has to be anonymized. Number two, we have to make sure we’re asking patients for their consent in order to use their data for research.
Rich: Yeah.
Kamal: And then number three, there’s a risk, when you look retrospectively that you are going to find a statistical fluke. It’s much easier to accidentally find something that turns out to be incorrect when you’re looking back.
Rich: Yeah.
Kamal: And so that can be a real issue with sort of real-world evidence.
Rich: Right.
Kamal: So making sure that we have appropriate statistical frameworks, for use of a wonky term, to make sure—
Paul: Oh, you’re in a safe space.
Kamal: [laughing] Okay. Good.
Paul: No one who listens to this podcast is going to—
Rich: Turn it off—
Kamal: Is going to bat an eye? Oh my gosh.
Paul: Who is this lady, this doctor with the statistical framework?
Rich: They haven’t been funny in a while.
Paul: Yeah, yeah, I think you’re good. You’re good.
Kamal: Okay, fine.
Paul: You’re very safe here.
Rich: Yeah, yeah, yeah.
Kamal: Because, you know, if the data is prospectively collected and appropriately controlled, that’s when we sort of have some level of confidence that we can trust the output from it.
Rich: Yeah.
Kamal: But if you’re just looking retrospectively, there’s so many things that change care over time. To isolate one variable and then say it’s because of this variable, we need to change our practice around this one way of doing things?
Rich: Yup.
Kamal: That can become very dangerous. So I think the use of data within one hospital or one hospital system will now be enabled. And I think there are certain things that you can learn in terms of like, quality improvement, quality assurance?
Rich: Yup.
Kamal: But in terms of actually changing the practice of medicine, I think it will be good for hypothesis generation, but then it’s going to inform the development of prospective clinical trials so that you can validate… [laughing]
Paul: This is the coldest bucket of water to pour on so many dudes right now. Like, there are so many dudes out there who are like, “We’re just going to aggregate all the medical data in the world. We’re going to feed it to the LLM and then billions of dollars will flow to me and I will get to have a Bugatti.” [laughter] And you are just here like, “Actually? Actually, fellas?”
Kamal: In this field…
Paul: I got some real bad news for you.
Kamal: It doesn’t quite work that way.
Paul: You’re not just gonna put those two spreadsheets together and then sell it to, like, to Cornell Weill. No, no, sorry. Bad news. Okay, that’s good. That’s good for everyone to know.
Kamal: Yeah, we have to make sure we have appropriate consent. We have to make sure the data is appropriately anonymized and cleaned. And then using a retrospectoscope and just looking back into the past, you’re not always going to get the right answer.
Paul: Doctors are the scariest managers because they will just, they’ll be like, “Well, there’s a couple million dollars spent, but, you know, what are you going to do?” [laughter]
Kamal: Darn!
Paul: Darn! Didn’t work!
Rich: Getting it wrong is not just a bug in software.
Kamal: Right.
Rich: It’s a different standard here, right?
Kamal: Yeah.
Rich: Let’s look ahead, then. First, I mean, there is a lot of, on the scientific side, rather than the clinical side, there’s obviously, there’s a lot of ambition around, and a lot of sort of motivation around, bringing these tools, which, by the way, are a lot better today than a year ago, two years ago.
Kamal: Yeah.
Rich: Are you optimistic about bringing these capabilities to bear for the research side, to sort of the to steal a term, moonshot side of things that isn’t just about scouring old data?
Kamal: You know, when I think about moonshot, I think a lot about public health interventions. There was a study that came out from the NCI, the National Cancer Institute, in December of 2024. And what they found is that prevention and screening efforts averted almost five million deaths from five different cancer types. And depending on the cancer type, efforts in prevention and screening actually far outweighed the benefit that you got from developing new treatments.
And, you know, there is some variability depending on the cancer type. But when we think about, you know, what are the things that we can do using AI that would really help us achieve the moonshot goals, a lot of it is thinking about how do we improve our public health and prevention efforts.
Rich: Mmm.
Kamal: So something as simple as smoking cessation, you know, tobacco control efforts accounted for 98% of the almost three and a half million deaths that were prevented from lung cancer. So if we could just, you know—
Rich: Yeah.
Kamal: Make better smoking cessation campaigns, remind people, find other ways to get the word out, you know, if we could increase screening for lung cancer, because lung cancer screening is far below what it should be in this country. If we could just increase lung cancer screening, we would avert so many deaths. So what are the ways that we can use AI to improve existing public health efforts? I think is going to be really important in terms of reaching those goals.
Rich: Interesting. And it’s counter to the, there is such a, what’s the silver bullet across every industry with AI? Every—AI really rewards laziness. [laughter]
Paul: You know what it is?
Rich: It really is, like, you know what, just throw it in the box. We’ll take care of the rest. Right? And I think everyone is getting burned over and over again, nowadays.
Kamal: Yeah.
Rich: Like, whether it be software that got deleted. You hear that like every couple of weeks, like, “Oh, yeah, just use AI to do that.” And it’s like, “Well, it just deleted the whole code base.” I think it’s interesting to hear this sort of, there’s value here. It’s incremental. Let’s be thoughtful about it. There is no big grand surprise.
Paul: It’s not a solution. It’s a tool.
Rich: It’s a tool.
Paul: And we had a really good podcast recently where we interviewed Andrew Leland, who’s a person who’s got very, very low vision, and he’s using AI very creatively. He wasn’t a technologist, and he’s kind of become one because it lets him have access to all sorts of stuff.
Rich: Built his own tools.
Paul: And I was excited for him. I was like, yeah! And he was like, “Hold on a minute. We didn’t quite solve it.” [laughter] Just going, “Hey there, buddy. Hey there. That sounds real—it’s nice you’re excited.”
Rich: “I can read the taco menu.”
Paul: “Yeah. But maybe, maybe we could have, like, you know, lots of equal rights and a national awareness campaign before you just get everybody Claude, okay?”
Rich: It’s a chipping away. Right?
Kamal: Right.
Rich: And I think the hard work is still, he still learned a lot of things to get this thing to be useful for his life.
Kamal: Yeah.
Rich: It wasn’t like he just sort of parachuted it in. He had to think about—he’s like, “I started making software. I’d never made software before, but I had these capabilities.”
Paul: But the framing is really interesting. I think it’s important framing. I really, like, I’m internalizing it because I love tech. I just do. And a lot of our audience does. But what we’re hearing over and over from people who are sort of in a field or really trying to solve something very, very difficult is that, wow, these are cool and exciting tools. I’m using them, people around me are using them. But there’s this one bright line where even if they say they have all the answers, it’s kind of meaningless. And it actually ties in with, hey, that’s nice that you think that we’re going to unlock all of this. But the reality is, if dad stops smoking, he’s going to live a lot longer.
Kamal: Mmm hmm.
Paul: And if you aggregate that, you get to five million people and that’s, like, a trillion dollars you’re not spending. And that’s a lot of lives that are saved and a lot of grandkids who get to know their grandparents. And so before you jump ahead to how we’re all going to live on Mars, maybe you could focus on the fact that with a couple really nice posters that we put around everywhere, we could save hundreds of thousands of lives. And I think hearing that is just, it’s such a hard framing to get across in this moment. I’m saying it, I’m doubling down on it because I think it’s just really hard to hear when several trillion dollars in motive, capitalism force is out there just saying—
Rich: Chasing everything.
Kamal: Yeah.
Paul: Saying, “No, no, don’t listen to her, she’s being very conservative. She’s a very nice doctor, but we’re going to fix this. We got it. Because we’ll just ingest more data.” And then you go like, “Yeah, good luck with that. Good luck with that. Because we’re going to have, if it’s from two hospital systems, it’ll probably tell you something completely wrong.”
Kamal: Right.
Paul: You’ve got that statistics degree. That’s just hurtful. [laughter] That’s just hurtful. Okay, new generation of doctors is coming in.
Kamal: Mmm hmm.
Paul: You have to—
Rich: Uh oh. Curveball closing question.
Kamal: Uh oh.
Paul: You have to get them to behave. They’ve been using this all through med school. Here they are, they’re yours now. You’ve adopted fledgling baby ducks following you around the hospital. [laughter] And they’ve got ChatGPT. What do you tell them?
Kamal: I think AI can be an incredible tool as long as you use it for learning and not to replace critical thinking.
Paul: Mmm. Okay.
Kamal: And so when I’m, you know, for example, I just finished a couple weeks in the hospital, I had residents with me, and we would pause during rounds and we would actually think, talk through the framework, talk through the structured thinking that went behind making a particular decision. And I would pause rounds and I would ask them questions about how they were thinking about this, how they were actually approaching this question. And they didn’t have time to look anything up, you know, I expected them to have already looked things up, to already have the knowledge walking in and for our conversation to be focused on how do you use that knowledge to think critically and make the right decision? And so I think it’s important that the that isn’t lost in all of this. AI can be a great tool for learning, for collecting more information, but we can’t let ourselves depend on it to make decisions that we should be critically thinking through.
Paul: So the process and the framework more than the output and the facts.
Kamal: Absolutely.
Paul: Okay. How are they, like, “…yeah, okay.” [laughter] Are they sneaking back to the text box?
Kamal: I got a lot of really positive feedback for that way of teaching. I think also again, and this is something that I really think a lot about in terms of healthcare and AI, that human connection is not to be undervalued.
Paul: No, I think people want it. We had a group of undergrads from the New School and they were lovely and they came in to learn about AI and they’re studying their sort of journalism adjacent. And we asked them what they were up to, and they were like, “Oh, we try to use it as little as possible. I’m going to a good college. I don’t want to waste that on having it answer everything.” And we were both kind of oddly, it’s funny, because here we are at our AI company, but it’s very inspiring.
Kamal: Yeah.
Paul: Yeah. They also have very, like, brightly dyed hair, [laughter] and they were like birds. It was wonderful. They were just great.
Rich: That’s a very thoughtful, no-escape framing of the question for your residents, obviously. Not everyone’s doing that. Do you worry at all about the new generation of physicians that are up and coming, who, let’s face it, they’re leaning on it maybe a little too much, and they’re not seeding the right experience over time because it’s an incredible crutch. It’s an incredible thing to lean on. Do you worry about it at all?
Kamal: I think it’s worth worrying about.
Rich: Yeah.
Kamal: There was actually a New England Journal article that talked about concerns that people would plateau their level of knowledge over time as a physician would plateau if they relied too heavily on AI.
Rich: Yeah.
Kamal: And actually went through frameworks that you could use in an educational setting to try to prevent that from happening.
Rich: Ah. Right.
Kamal: So this is an ongoing discussion. I think a lot of people are concerned about this.
Rich: Are concerned, yeah.
Kamal: I will also say, when you go from being in training to actually being an attending, and you are the one responsible for making that decision for your patient.
Rich: High stakes.
Kamal: There is nothing scarier than the idea that you might make the wrong decision and hurt somebody.
Rich: Sure.
Kamal: Or you may not give them as much time on this earth as they could have because of a decision that you made.
Rich: Sure, sure.
Kamal: And so that, again, for me, it comes back to, what is the human part of interacting with AI? And so as a person who wants to do their best taking care of another person, you’re going to do everything you can to try to make sure you’re making the right decisions.
Rich: Sure.
Kamal: And so whether you’re using the information from AI or not, you’re probably still going to go to your peers, you’re probably going to go to senior attendings and say, “Am I making the right decision? Am I thinking about this the right way?”
Rich: Sure.
Kamal: So we may sort of see, you know, sort of a delayed period of training as people ask more questions and learn. But once people have to develop that independence, I think they’ll realize the importance of being able to critically think through things and not just relying on the AI.
Paul: I gotta tell you, too, you know, I get a little sense of, like, “Dr. Menghrajani’s students would never.” [laughter] It’s not, it’s just not happening.
Rich: Don’t go there.
Paul: No, I know you get the sense. It’s all very, very nice, but you also get the sense that there’s a steely, steely glare that can happen, and it’s not to be trifled with.
Kamal: We do our best to make sure that those who are in training are well prepared to take exceptionally good care of their patients when they’re done. And so, you know, that requires—
Rich: The bar is high.
Kamal: The bar is high. They know it’s high, the expectations are set, and they meet them.
Paul: Well, this has been amazing. Thank you very much. We’ve learned a lot. Have you learned a lot?
Rich: Yes. It’s good.
Paul: It’s good. Good to see something get in there.
Rich: Great conversation. Really interesting time. I feel like if we chatted in a year, we’ll have a whole new set of topics to talk about. It’s moving so fast.
Kamal: Yeah. Absolutely. Liability and regulation and who’s paying for this.
Rich: [laughing] Oh!
Kamal: And AI and drug development. And yeah, there’s a lot more to talk about all of that.
Rich: Yeah. Tons more.
Paul: All of it, but let me summarize it this way: Don’t count on AI to replace your doctor. Use it to learn about public health.
Kamal: [laughing] Yes!
Paul: There we go.
Kamal: More people should learn about…
Paul: Go ask Claude about good smoking-cessation programs globally and sort of how that affected cancer rates. Let it draw you a graph, right? That’s a thing we should all be doing.
Kamal: You know, doing the things that your mom told you were good for you and making sure you’re staying consistent with those things. Getting outside, getting lots of sunlight. You know—
Paul: I think Rich’s mom handed him a pack of cigarettes. [laughter]
Rich: My mom told me to do that today.
Paul: Yeah.
Rich: Oh, that’s true. Cigarettes, yes.
Paul: Oh, yeah, absolutely.
Rich: Yeah.
Kamal: Sleeping well, eating properly, getting your vegetables, you know, social interaction. These are the things that build health and wellness over a lifetime. And so whether or not AI is where you get that information from, make sure it’s a trusted source. You know, not trying to sell you supplements or something, but actually trying to help you live the healthiest, best life that you can.
Rich: Great ending. I mean, turns out mom was right.
Kamal: Yes, absolutely.
Rich: Turns out mom was right. Reach out. Hello@aboard.com. Show topics? Need help? Check us out at aboard.com.
Paul: Dr. Menghrajani, if anybody wants to sort of get in touch with you or sort of learn about you, where should they go?
Kamal: You can find me, I have a page, it’s called Health Insights LLC. And so you can just Google that domain and you’ll find me there.
Paul: Great. We’ll put it in the show notes.
Kamal: Thanks.
Paul: Thank you so much for coming on.
Kamal: Thank you. Thank you so much for having me. This was a blast.
[outro music]