CRASH LAB (@CRASHLabAI) header
CRASH LAB (@CRASHLabAI) profile picture

The Centre for Responsible Autonomous Systems in Healthcare is a clinician-led collaborative research group anchored at KCDH-A

Joined June 2026
Photo by CRASH LAB (@CRASHLabAI)Photo by CRASH LAB (@CRASHLabAI)Photo by CRASH LAB (@CRASHLabAI)Photo by CRASH LAB (@CRASHLabAI)Photo by CRASH LAB (@CRASHLabAI)Photo by CRASH LAB (@CRASHLabAI)Photo by CRASH LAB (@CRASHLabAI)Photo by CRASH LAB (@CRASHLabAI)
🚨 Calling clinicians across India! Join #PrAImaan #ValiDATAthon to evaluate AI-generated clinical responses. 🩺 🏅 Hands-on training | Certification | Research opportunities 📍 Ahmedabad, Tumkur, Mysuru, Bengaluru, Eluru & Puducherry 📲 Scan the QR to register!
1
2
125
As Lead, Medical Benchmarks at CRASH Lab, KCDH-A, Sarah champions responsible healthcare AI grounded in scientific rigour. she builds bridges between medicine, technology & ethics. Here's to a wonderful year ahead. Enjoy your day, Sarah! 🎂 Team #CRASHLab @saranahisarah
3
1,206
#ThoughtLeadership | Our Founder @DrDatta_AIIMS was featured on The ElevAItor Ride with Walter Robinson on the promise & peril of #AIinHealthcare. Dr. Datta's advice: ask for evidence 🧾 and use AI as a companion to your doctor, not a replacement. Read more 👇 @AI_4_Healthcare
2
5
481
Thank you to my friend @HealthcareAIGuy for the feature and honoured to be a passenger on The ElevAItor Ride, his #AIinHealthcare newsletter, on the day it turns one year old. Coincidentally, I myself turned a year older yesterday! Three things I talk about: 1. The promise on how AI could finally take high-quality care to the last mile, to rural India and other LMICs. 2. The peril of models flooding healthcare with big claims and little evidence and how this could widen inequity. 3. The path towards trusting any AI tool before use by asking two simple questions: Has it been tested on people like me? Has it been tested in my language? Congratulations on a year of building this global conversation! @CRASHLabAI @KCDH_A
2
4
11
1,206
🚨Post NEET PG results, I have been getting DMs from genuinely confused young doctors. As we witness AI making serious inroads into healthcare, we know the current system won't look the same after the commitment of 3 years folks are about to make. 👇🏽 As promised, wrote a short blog on choosing residency. It is a 2 minute read. Opus 5.5 made a nice music video too! 🫰Hopefully this one helps clear some doubts! :)
11
14
4
77
22,539
CRASH Lab @KCDH_A is looking for full-time Clinical Research Leads to join us on-site in 📍Sonipat. Work on cutting-edge projects reshaping the future of AI-driven #healthcare and make a nationwide impact. 📝 Apply here: forms.gle/sV1TZoTXdo45xMML7 #AIinHealthcare #DigitalHealth
1
1
6
1,057
CRASH Lab @KCDH_A is calling for clinical experts to join our various National Clinician-led Initiatives for AI in Healthcare. Work directly with us to evaluate frontier AI models & help ensure their clinical safety & accuracy. Express interest 👉 tally.so/r/pbYa1y
1
3
854
Happy birthday to Sahaj Vaidya, our Senior Research Associate at CRASH Lab, KCDH-A, & Visiting Researcher at the Laboratory for intelligent Global Health and Humanitarian Response Technologies (LiGHT), EPFL. 🥳
1
561
The team at #CRASHLab would like to wish a very happy birthday to our Project Assistant, Vandna. 🥳🎂
1
1
4
508
Happy birthday to the sincerest member of the team! 🥳🌸
2
1
40
Celebrating @svalecha98 🎉 Today we’re wishing a very happy birthday to Sid, Product Manager at CRASH Lab.
1
2
6
845
⚡️Our 4th metric: RadLE-S = Safety Index. This measures protection against confident wrong diagnoses and one of the most important ones! A wrong answer with high confidence is very dangerous and RadLE-S penalizes wrong answers more heavily when confidence is high. 🏆 Top AI models on RadLE-S: 1. Claude Fable 5 — 56.8 👑 2. OctoMed 7B — 52.3 3. Meta Muse Spark 1.1 — 48.0 Human expert baseline: 67.0 This is where some models really suffer as they make mistakes confidently. A confident wrong answer can mislead a trainee, reassure a patient falsely, or push a clinician toward the wrong diagnosis. That is why safety needs its own metric. cc: @AnthropicAI @MetaAI @VPrasadMDMPH @HealthcareAIGuy @WillyRontgen @francisdeng @anish_koka @RoupenMD @venkmurthy @zakkohane @arjunmanrai @AdamRodmanMD
1
3
17
2,020
⚡️Our 3rd Metric (and everyone's favourite): RadLE-A = Accuracy Index. ✅This is the usual benchmark question: Out of 200 cases, how many did the model get right? 🏆 Top AI models on RadLE-A: 1. Gemini 3.1 Pro — 32.0 👑 2. GPT-5.6 Sol Pro — 29.0 3. Meta Muse Spark 1.1 — 28.5 Human expert baseline: 38.5 Gemini 3.1 Pro was the most accurate AI model and maintains its leadership in accuracy from RadLE 1.0, despite being many months old (We wait for 3.5 Pro eagerly). Because autonomous diagnosis is not just about getting more cases right. It is also about whether the model knows when it might be wrong. cc: @GoogleDeepMind @demishassabis @JeffDean @vivnat @alan_karthi @GoogleAI @demishassabis @OpenAI @sama @gdb @MetaAI @Andrejkarpathy @rohanpaul_ai @kimmonismus @haider1 @pushmeet @OfficialLoganK
1
5
19
1,874
⚡️Our 2nd metric: RadLE-R = Reliability Index. This asks a very simple question: When the model gives an autonomous-level answer, is it actually right? Likert 3–4 = Autonomous-level output. Likert 0–2 or “I don’t know” = Hand over to human specialist. 🏆 Top AI models on RadLE-R: 1. Claude Fable 5 — 54.7 👑 2. Meta Muse Spark 1.1 — 40.2 3. Gemini 3.1 Pro — 32.5 Human expert baseline: 55.0 This was one of the strongest results. Claude Fable 5 was almost exactly at the human expert baseline on reliability of autonomous-level outputs. But most models dropped sharply. So yes, models are getting more accurate. But their confidence is still not consistently trustworthy. And if confidence cannot be trusted, autonomy cannot be trusted. cc: @AnthropicAI @MetaAI @GoogleDeepMind @demishassabis @DrEliDavid @VPrasadMDMPH @hvanspall @AnilMakam @rsumbaly @AIatMeta
1
3
15
1,665
⚡️The first and primary metric of RadLE 2.0 is RadLE-C = Confidence Weighted Index. >Correct + confident = rewarded. >Wrong + confident = penalized. Because in medicine, a confident wrong diagnosis is not just an error. It is a dangerous error. 🏆 Top AI models on RadLE-C: 1. Claude Fable 5 — 758 👑 2. Meta Muse Spark 1.1 — 735 3. GPT-5.6 Sol Pro — 665 ✅Human expert baseline: 988.7 So the best AI is still 231 points behind the average human expert baseline. 🚨The key finding: The most accurate model was not the top model overall. Once confidence entered the equation, the leaderboard changed. cc: @AnthropicAI @MetaAI @OpenAI @karpathy @sama @gdb @tibo_maker @alexandr_wang @Dr_Singularity @rohanpaul_ai @DrEliDavid @reach_vb @thsottiaux @DarioAmodei @jackclarkSF @ThomasScialom @giffmana @kimmonismus @thekaransinghal
2
4
19
2,328
🔥Today, we are releasing one of the first visual reasoning benchmarks for autonomous AI diagnosis in healthcare! 🚀Introducing Radiology’s Last Exam 2.0 (RadLE 2.0) from @CRASHLabAI, an uncertainty-aware benchmark for autonomous diagnosis in radiology! ✅In the last few days, the AI frontier has moved significantly. @OpenAI launched GPT-5.6 Sol. @Meta launched Muse Spark 1.1. @xAI dropped Grok 4.5. 🙌We’ve benchmarked all frontier, open-source and medical VLMs in RadLE2.0 and the leaderboard is now LIVE! 🚨 Before AI models are handed autonomy, one question matters more than any accuracy score: Do they know when to STOP and hand over to a human? ⚠️ A confident wrong diagnosis is far more dangerous than an honest “I don’t know.” Yet most models are bad at admitting the latter! 🚀 We release five RadLE 2.0 Scores: Confidence Weighted, Reliability, Accuracy, Safety and Handover Readiness and we find that models from @OpenAI @AnthropicAI @MetaAI @GoogleDeepMind @xAI @nvidia @Alibaba_Qwen @MistralAI @MiniMax_AI all score very differently as they optimize for different metrics! 🚨But most importantly, NONE of the Models have been able to reach the average human expert baseline! ⚡️A thread on what we found and which models aced our metrics! Link to the leaderboard and technical report at the end of the thread!
26
49
19
275
185,984
CRASH LAB retweeted
🔥TL;DR: Radiology's Last Exam 2.0 releasing this week! Stay tuned! ✅People are increasingly using frontier AI models for healthcare diagnosis. And the promise of autonomous AI systems in healthcare has also surfaced recently. 🚨But unchecked social media hype around AI-reported diagnoses is creating a blind trust in models not cleared by any regulatory body. 🤔Just because your AI chatbot sounds confident does it mean it is accurate? 🙌At @CRASHLabAI, we have been actively asking: Are we truly ready for autonomous healthcare systems? With Radiology's Last Exam (RadLE) 1.0 last year, we tested whether the speculation about doctors becoming obsolete actually holds up. Today's AI genuinely works for many common presentations. Where it falls behind are the rare ones! 🔥This week, as the @AIforGood conference commences in Geneva, with world's AI leaders flying in, we are releasing RadLE 2.0 to benchmark the readiness of autonomous AI agents for radiologic diagnosis. 🚨We believe that a model which gives wrong answers confidently is far more dangerous in medicine than one that admits uncertainty! This week, we are going to share with you whether human beings or AI models win this battle. :) ✅RadLE 2.0 is coming… and you will surely want to see these results! Follow @CRASHLabAI and stay tuned! 🇨🇭P.S. If you are in Switzerland for the Summit or around @EPFL or @ETH Zurich for the next 2 weeks, I will be here and happy to meet! #RadLE #HealthcareAI #RadiologyAI
2
2
258
🔥TL;DR: Radiology's Last Exam 2.0 releasing this week! Stay tuned! ✅People are increasingly using frontier AI models for healthcare diagnosis. And the promise of autonomous AI systems in healthcare has also surfaced recently. 🚨But unchecked social media hype around AI-reported diagnoses is creating a blind trust in models not cleared by any regulatory body. 🤔Just because your AI chatbot sounds confident does it mean it is accurate? 🙌At @CRASHLabAI, we have been actively asking: Are we truly ready for autonomous healthcare systems? With Radiology's Last Exam (RadLE) 1.0 last year, we tested whether the speculation about doctors becoming obsolete actually holds up. Today's AI genuinely works for many common presentations. Where it falls behind are the rare ones! 🔥This week, as the @AIforGood conference commences in Geneva, with world's AI leaders flying in, we are releasing RadLE 2.0 to benchmark the readiness of autonomous AI agents for radiologic diagnosis. 🚨We believe that a model which gives wrong answers confidently is far more dangerous in medicine than one that admits uncertainty! This week, we are going to share with you whether human beings or AI models win this battle. :) ✅RadLE 2.0 is coming… and you will surely want to see these results! Follow @CRASHLabAI and stay tuned! 🇨🇭P.S. If you are in Switzerland for the Summit or around @EPFL or @ETH Zurich for the next 2 weeks, I will be here and happy to meet! #RadLE #HealthcareAI #RadiologyAI
1
18
3
53
19,669
This is exactly the kind of story that makes better evaluation important. AI is becoming increasingly capable in common cases. But the question is how it performs on the ones that are uncommon? That's the question RadLE 2.0 is built to answer. Results out soon!
Exactly why we need to release Radiology’s Last Exam v2 Congratulations to Gaurav. AI models are really good at diagnosing the common and acute conditions. These are 90% of cases. Hence people talk about this on social media more. But they fail when they move towards rarer conditions, something we call as Out of Distribution (OOD) cases. We must make the public aware of these limitations otherwise they start believing AI is always correct. RadLE v2 needs to be out quick. And we are working hard to get this out. More contributors (radiologists) are welcome! P.S. In next version we plan to expand to all visual domains (beyond radiology).
3
1
9
3,259