PhD @ UWaterloo, former intern @llama_index. Focused on LLMs and RAG, especially to understand and explain their behaviors.
Waterloo, Ontario
Joined May 2023
- Tweets15
- Following87
- Followers36
- Likes40
Last week at #EDBT2025, I presented our latest demo, LADYBUG — an interactive debugger for LLM agents 🐞
LLM agents are powerful, but when they fail, debugging can be a nightmare. LADYBUG helps you trace agent steps, intervene, and even repair issues with LLM assistance.
🧵👇
LADYBUG is like a software debugger, but built for LLM agents.
🔍 Step-by-step execution traces
🛠️ Intervene at any step & re-run only what’s affected
🤖 LLM-powered self-reflection to find & fix errors
📈 Inspect intermediate state & metrics in complex workflows
For more details, please see our paper and demo video:
📄 Paper: openproceedings.org/2025/con…
🎥 Demo video: vimeo.com/1036113298
Thanks to my co-authors Parke Godfrey, Lukasz Golab, Divesh Srivastava, and Jarek Szlichta!
#AI #LLM #DevTools #Debugging #Agents
Joel Rorseth retweeted
🎊As we ring in the New Year, check out our 2024 roundup story!
🌟From revolutionizing healthcare to unravelling the secrets of AI, #UWaterloo's Computer Science is pushing the boundaries of human curiosity.
🔗Read more: cs.uwaterloo.ca/news/recap-2…
Had a great time chatting with @WCDSBInnovates on their Deep Learning Dialogues podcast! We discussed my recent LLM-focused #ExplainableAI research and its implications for AI trust and safety. Check it out! 🎙️
New episode! 🎧 "RAGE: Bringing Clarity to the Fuzzy World of AI Sources" featuring @JoelExplainsAI from @UWaterloo uncovers how the RAGE tool demystifies LLM sources, reshaping AI trust in education. Listen now on all podcast platforms or our website! #WCDSBInnovates
Check out this great article about RAGE, our latest research innovation that helps explain LLMs!
A team of #UWaterloo researchers have created a new tool – nicknamed “RAGE” – that reveals where large language models (LLMs) like ChatGPT are getting their information and whether that information can be trusted.
More: bit.ly/3RfAJm2 | #UWaterlooNews
ALT A Chat GTP Hologram as someone types on their phone
6/ 🧑🔬 This work is a collaboration between @UWaterloo, @YorkUniversity, and @ATT. Huge thanks to my co-authors for their incredible contributions! 🙌
Ever wonder how exactly your knowledge sources are being used during RAG?
We did too. I'm excited to share our early work in explaining LLMs, "RAGE Against the Machine: Retrieval-Augmented LLM Explanations"! 🎸✨
📄 Paper: arxiv.org/abs/2405.13000
🧵 [Thread]
5/ 💻 Dive into our demo paper to explore the technical details and implications of our findings. Link: arxiv.org/abs/2405.13000 🌐📖
4/ 📈 Whether it's determining the greatest tennis player or answering your own unique questions, RAGE supports various use cases. Our code (coming soon) is designed with interfaces, making it highly adaptable to any LLM, retriever, or text data source! 🎾🏆
3/ 🌟 Key features include:
- Source Combination Tests: Which sources lead to which answers? 🔍
- Source Permutation Tests: How does the answer change if sources are reordered? 🔀
- Interactive Demo: Explore behaviors of real LLMs using real data. 📚
2/ 🛠️ RAGE helps users understand how LLMs generate answers by identifying parts of the input context that, when moved or removed, change the LLM's response. This counterfactual approach makes AI decision-making more transparent, and allows us to derive a form of citation. 🔄📊
1/ 🔍 Our paper introduces RAGE, an interactive tool designed to explain the outputs of LLMs augmented with retrieval capabilities (RAG). RAG needs explaining since it is unclear how the presence and order of knowledge sources (among other things) affects the LLM's answer! 🤖💡
Joel Rorseth retweeted
Being able to interpret an #ML model’s hidden representations is key to understanding its behavior. Today we introduce Patchscopes, an approach that trains #LLMs to provide natural language explanations of their own hidden representations. Learn more → goo.gle/4aS5epd
ALT Illustration of our framework, showing a Patchscope for decoding what is encoded in the representation of “It” in the source prompt (left), by using a predefined target prompt (right).