Building tools for AI devs @nvidia // CS @ Berkeley, GA Tech // Sharing what I'm learning
UC Berkeley
Joined January 2015
- Tweets167
- Following696
- Followers202
- Likes1.3K
Pinned Tweet
CacheFuse is an early attempt in cross-model KV cache reuse that demonstrates that linearly interpolating the KV caches of two models can outperform a standalone model’s cache alone. I’ll be presenting it tomorrow at #COLM Efficient Reasoning Workshop - stop by!
adding CacheFuse (@rishiskhare, COLM 2026 workshop) to the latent communication series, because it's the simplest version of the idea I've seen work
Qwen3-8B and Qwen3-4B have KV caches of exactly the same shape, so you can just blend them (roughly half and half) and let the 4B answer from the mix so there's no training at all
the 4B gains up to 8.6 points and beats the standalone 8B on all four benchmarks (best mix picked per benchmark, though a held-out calibration lands close)
however this trick needs siblings with matching cache shapes and a shared tokenizer, while e.g. models different by x200 is sizes and from different families have nothing to blend, which is the case we train a bridge for at @mostik_ai 🌉
Rishi Khare @ NeurIPS 2026 (Atlanta) retweeted
adding CacheFuse (@rishiskhare, COLM 2026 workshop) to the latent communication series, because it's the simplest version of the idea I've seen work
Qwen3-8B and Qwen3-4B have KV caches of exactly the same shape, so you can just blend them (roughly half and half) and let the 4B answer from the mix so there's no training at all
the 4B gains up to 8.6 points and beats the standalone 8B on all four benchmarks (best mix picked per benchmark, though a held-out calibration lands close)
however this trick needs siblings with matching cache shapes and a shared tokenizer, while e.g. models different by x200 is sizes and from different families have nothing to blend, which is the case we train a bridge for at @mostik_ai 🌉
You can use your favorite CLI coding agent to control live Chrome tabs now.
An integrated terminal within Chrome to automate browser use.
Completely free and open-source: github.com/rishiskhare/agent…
100% agree on the Context Hub. Developers constantly tweak their approaches to manage context in their prompts with each new model and tool suite release. I would even say that the core problem we face is a "Config Hub" problem - different configurations of models, harnesses, and data augmentation techniques are making managing agents in prod much more brittle and difficult.
Rishi Khare @ NeurIPS 2026 (Atlanta) retweeted
Excited to share that my ICLR 2026 Oral Talk for GEPA is available on YouTube.
I go deeper into why GEPA works better than prior optimization techniques, along with touching on many aspects of GEPA!
youtu.be/HbGah-uP1fI
Thrilled to present GEPA as an Oral Talk and Poster at ICLR 2026 this Friday in Rio! 🇧🇷
Apr 24
Oral Session 3A (Agents), 10:30 AM BRT, Amphitheater
Poster Session 4, 3:15 PM, Pavilion 3
teapawt.net/LakshyAAAgrawal/status…
Let's recap what's happened since we released GEPA last year 🧵
Rishi Khare @ NeurIPS 2026 (Atlanta) retweeted
Very excited to share that GEPA is now live on @DSPyOSS as dspy.GEPA!
This is an early code release. We’re looking forward to community feedback, especially about any practical challenges in switching optimizers.
Rishi Khare @ NeurIPS 2026 (Atlanta) retweeted
🤯 must watch
GEPA is a SUPER exciting advancement for @DSPyOSS and a new generation of optimization algorithms re-imagined with LLMs! 🧩🚀
Starting with the title of the paper, the authors find that Reflective Prompt Evolution can outperform Reinforcement Learning!! 🤯
Using LLMs to write and refine prompts (for another LLM to complete a task) is outperforming (!!) highly targeted gradient descent updates using cutting-edge RL algorithms! ⚖️
GEPA makes three key innovations on how exactly we use LLMs to propose prompts for LLMs -- (1) Pareto Optimal Candidate Selection, (2) Reflective Prompt Mutation, and (3) System-Aware Merging for optimizing Compound AI Systems. 🧠🧠
The authors further present how GEPA can be used for training at test-time, one of the most exciting directions AI is evolving in! 🚀
Here is my review of the paper! I hope you find it useful! 🎙️
Rishi Khare @ NeurIPS 2026 (Atlanta) retweeted
Guy has his fingers positioned wrong on his keyboard and types gibberish into ChatGPT.
It decodes what he meant to say by assuming his hand was shifted right and re-mapping his fingers 🤯
(from u/mimic751)
Rishi Khare @ NeurIPS 2026 (Atlanta) retweeted
Iterative reflections for LLMs can outperform heavy RL?
This paper shows that having the LLM reflects on its own trajectories, rewrite its own prompts, and evolve a diverse pool of candidates beats RL w/ GRPO so far on four reasoning tasks .
10% improv with 35x fewer rollouts!
Rishi Khare @ NeurIPS 2026 (Atlanta) retweeted
Infrastructure for Multi-Agent Systems
@koomen
Multi-agent systems are powerful but hard to build. Think agentic MapReduce with thousands of subagents running in parallel. We're looking for folks who've felt the pain of scaling these systems and want to make operating fleets of agents as easy as deploying a web service.