prof @MIT CSAIL | ColBERT, https://xitter.cf/t.co/ZZaSzaRaZ7 (@DSPyOSS), GEPA, RLMs, Pedagogical RL

Cambridge, MA
Joined December 2022
confession: i've yet to have that moment of feeling that frontier LLMs are geniuses, or super-human at ~anything broadly coherent that i deeply prize
51
10
5
398
22,057
i'm not sure what it is that my work needs or what i'm presumably 'good' at, but despite extremely elaborate (& happy!) use and having worked on nothing but language models for essentially my entire career, i can only see that kind of vibe abstractly from others' (e.g., mathematicians) reactions, not in the vicinity of anything that i need or do they're the most impressive tools i've ever used, and i essentially use my computer mostly from them now, but yeah, not yet anyway, back to cc and codex cuz we have work to do
4
48
3,723
one way to understand what i mean is the sheer amount of expertise AND (expert) effort required to extract high-caliber work out of them - i guess all the slop research complaints out there are a common consequence of this
8
56
3,133
Omar Khattab retweeted
It's like self-supervised learning at the higher abstraction of research contributions instead of words: take a naturally occurring thing, mask out a part, see if the AI can recover it. What's the next level up of abstraction? "Pretend we know nothing about the field of acoustics; what can you recover?" Congrats to @MangQiuyang, @lihanc02, and team. Another @LaudeInstitute Slingshot we're proud to have supported.
FrontierCS 2 is our new benchmark for CS research & development. Each task is based on a paper: we delete the contribution from the released code, and ask agents to outdo the original method, as judged by the authors' chosen metrics. Agents don't have access to the paper or web.
3
1
17
1,451
Omar Khattab retweeted
On this note, if you've ever thought "Man, I wish I had BEIR and NanoBEIR, but considerably better and very discriminative of model quality!", you might also want to stay tuned. i hear it might also be fully open
but you do need to evaluate them better, BEIR considered harmful (but we thank it for its services)
3
2
37
1,952
Omar Khattab retweeted
but you do need to evaluate them better, BEIR considered harmful (but we thank it for its services)
extremely excited for our upcoming releases. retrievers aren't even vaguely close to anything resembling a ceiling
2
1
1
24
3,492
Omar Khattab retweeted
extremely excited for our upcoming releases. retrievers aren't even vaguely close to anything resembling a ceiling
2
2
1
37
3,317
Omar Khattab retweeted
We've released a technical report on the released late interaction models It's purposely kept short, simple and direct, because most of the recipe is not new (shared space is not really new as well, but pretty cool for ColBERT still) There are still a lot of technical information if you are interested beyond the BP We are preparing much more thorough papers for things where we did really bring a lot of novelty, pretty sure you'll enjoy those! arxiv.org/pdf/2610.29523v1
gm chat 👑
1
7
91
2,683
Omar Khattab retweeted
Here we have a minimal tech report on 2 Colbert models:
gm chat 👑
1
4
43
2,154
Save 50%, only on Oct 21st! Today, you can save on Building LLM Applications with DSPy by Serj Smorodinsky and Brett Kennedy — now in print — and other related titles: hubs.la/Q04BMk270 Learn how to build DSPy into fully agentic pipelines, integrating conversation memories, and connecting agents to external environments via MCP.
2
1
17
1,686
Omar Khattab retweeted
applications for slingshots more than doubled from batch 3 to batch 4. funding open science is critical to democracy. never in the past have scientists had to turn down so much money to do research they can talk about openly. top PhD students are receiving offers of millions of dollars per year from closed labs. i'm proud that at Laude we support open research. great work on slingshots batch four @bradenhancock and the @laudeinstitute team
Introducing Slingshots // FOUR: 14 projects across eight top institutions. Self-improving agent systems and harnesses, maximally-open models and training data, applications for research and decision-making – plus kernels and more. Say hello to the batch 🧵
2
8
1
48
4,504
Omar Khattab retweeted
No one changes the world alone Introducing Ampersand - the first model built for us. Ampersand is a collaborator trained to help people achieve their individual and collective goals. In multi-month simulations, users collaborating with Ampersand accomplished more than with Fable and other frontier models Available to everyone through our API
98
122
99
1,118
226,602
Omar Khattab retweeted
Incredibly grateful to @LaudeInstitute for supporting our work on WaggleHaus! TL;DR: We believe that supporting agentic queries over massive data lakes requires rebuilding the data system from the ground up. This includes ideas ranging from: - making the data layer more "agent-friendly" - to specializing the inference engine for the query workload - to running agents within the system that continuously update views and indices for faster data access If you're interested in a demo or becoming an industry design partner you can sign up here: wagglehaus.ai
Replying to @LaudeInstitute
WaggleHaus // @RussoMatthew, @tianyu_li_, @MikeCafarella, @tim_kraska, @samrmadden (@MIT, @UWMadison) — A query engine for executing semantic and agentic queries over enterprise-scale data lakes.
2
6
1
21
1,360
Check out our latest research: MemCurrent, a persistent agent memory architecture, which consistently stays on the cost-accuracy Pareto frontier on our internal and three public benchmarks. Try it in the new Agent Bricks Memory and Genie 🚀
Introducing MemCurrent, a persistent memory architecture designed to help production agents stop starting from zero. MemCurrent combines episodic memory with derived memory across three access patterns: • Plugins to log and load context automatically • Tools the agent calls on demand • An offline “dreaming” pipeline that consolidates memory in the background Agents carry forward user preferences, prior corrections, metric definitions and successful workflows instead of rediscovering them every session. On our internal benchmark built from real enterprise data, MemCurrent triples the agent's pass rate while using 31% fewer tokens and running 26% faster. It's built into Agent Bricks' Memory offering and already powers memory in Genie. Read the research: databricks.com/blog/memcurre…
2
5
28
2,302
Omar Khattab retweeted
ScholarCatalyst is part of Slingshots Four! Deeply “reading” the literature, i.e. knowing which earlier idea a new problem needs, takes years to get even okay at. Even the best systems do poorly on ScholarCatalyst. With @LaudeInstitute's support, we’ll train directly for this.
Replying to @LaudeInstitute
ScholarCatalyst // @yoonholeee, @ohmyksh, @chelseabfinn (@Stanford) — A benchmark for finding prior work that could inspire scientific research, and a retrieval system trained for this task.
2
6
25
1,910
Pedagogical RL // @SOURADIPCHAKR18, @NoahZiems, @amritsinghbedi3, @lateinteraction (@MIT, @LaudeInstitute) — A reinforcement learning framework that teaches AI models to generate plausible, successful reasoning trajectories they can learn from for challenging tasks.
1
5
24
1,472
Headlong // @NickAJalbert, @bradenjhancock, @NoahZiems, @a1zhang, @lateinteraction, @andykonwinski (@LaudeInstitute, @MIT) — A microharness for persistent agents.
1
1
11
1,067
Slingshots // Four is live! Another batch, another fourteen projects shipping open source artifacts that I’m excited about. See below for the full list. A few themes from the batch: 1. Harness time: This quarter harnesses shot up in visibility, with increased appreciation for the parts of an agent that aren’t the model and what they can give you. This batch includes a harness supporting persistent agency (Headlong); a harness supporting self-improvement via context management (ACE); a harness supporting self-improvement of the agentic harness itself (Exo); and a harness supporting multi-agent setups (dotagents). 2. Up and down the stack: There’s innovation to be had at every layer of the stack. We saw projects worth backing at the level of kernels (Thunderkittens); inference (M*); training (Marin); data (Atlantic-CC); evals (Decision Assistants); optimization (SkyDiscover); applications (Wagglehaus), etc. 3. Talk of the town: Slingshots continue to tackle timely problems at the frontier. Concurrent with Jev taking over X, an even faster type 1 decision model (CLM) came out of academia. With RSI on the brain of every big lab researcher, we now have a public literature retrieval benchmark grounded in authors’ firsthand accounts of their research process (ScholarCatalyst). And a new RL training method (Pedagogical RL) addresses head-on the question vexing RL practitioners of how privileged information can be used to discover generalizable behaviors, not just verify. Congrats to all the researchers in this batch and the previous three who continue to ship their research out into the world with open artifacts that can be adopted and built upon. Open science leads to better outcomes for everyone. And the good times will continue to roll – batch five is looking just as good. If you’re a researcher with a project that’s gaining momentum and ready for an adrenaline shot, shoot me a DM or apply at laude.org/slingshots. 👏 @a1zhang, @AlexKrentsel, @ankrgyl, @Azaliamirh, @bariskasikci, @bfspector, @chelseabfinn, @chipro, @Diyi_Yang, @dlwh, @drmapavone, @HazyResearch, @jackyk02, @jonasgeiping, @JonSaadFalcon, @jure, @kothasuhas, @lateinteraction, @martin_casado, @mertcemri, @michaelryan207, @MikeCafarella, @NickAJalbert, @NoahZiems, @nsagan271, @ohmyksh, @percyliang, @qizhengz_alex, @RussoMatthew, @samrmadden, @shagarw21, @ShashwatGoel7, @shulynnliu, @simran_s_arora, @souradipchakr18, @stuart_sul, @tianyu_li_, @tim_kraska, @WilliamBarrHeld, @yoonholeee, @YejinChoinka, @Stanford, @MIT, @LaudeInstitute, @CarnegieMellon, @UW, @UCBerkeley, @MPI_IS, @UWMadison
Introducing Slingshots // FOUR: 14 projects across eight top institutions. Self-improving agent systems and harnesses, maximally-open models and training data, applications for research and decision-making – plus kernels and more. Say hello to the batch 🧵
7
12
2
59
10,445
ScholarCatalyst // @yoonholeee, @ohmyksh, @chelseabfinn (@Stanford) — A benchmark for finding prior work that could inspire scientific research, and a retrieval system trained for this task.
2
2
1
18
3,196
Omar Khattab retweeted
I'm giving a Rising Star talk on text optimization and "cumulative learning" at the AI Symposium at UMich today! (10:15AM ET)
⏰ Happening TOMORROW! The 2026 #MichiganAI Symposium brings together researchers, industry leaders & students to explore foundational models & their impact on academia, industry and society. 🎤 Keynotes, talks, posters & more, join us at @UMichCSE! ai.engin.umich.edu/events/20…
2
7
62
2,356