Assistant Professor at CMU. Marathoner, @thesisreview.
Joined April 2011
- Tweets1.5K
- Following241
- Followers7.7K
- Likes2.6K
Sean Welleck retweeted
The next few days are going to be exciting!
It's L3 Lab CUA Week!
Over the next week, we'll have several research releases about realtime agents that interact with the digital and physical worlds.
Stay tuned if you're interested in realtime agents, multi-agent communication, RL envs, computer-use, robotics, or new settings where frontier models score below 10%!
Sean Welleck retweeted
Today we’re bringing Aristotle to software and hardware!
Describe the properties your code must satisfy and Aristotle formally verifies that it meets them.
I’ve spent the past few months helping Aristotle expand from proving math to verifying code. I’m excited to see it available to everyone today.
Try it: aristotle.harmonic.fun
It's L3 Lab CUA Week!
Over the next week, we'll have several research releases about realtime agents that interact with the digital and physical worlds.
Stay tuned if you're interested in realtime agents, multi-agent communication, RL envs, computer-use, robotics, or new settings where frontier models score below 10%!
Led by Pranjal Aggarwal (@PranjalAggarw16)!
Follow along here and at our CUA Week page:
- cmu-l3.github.io/cua-week/
We'll update the page as each project is released.
Sean Welleck retweeted
Lean Verified Tokenizers (srush.github.io/verified-tok…)
Training-inference mismatch kills RL runs, but yet we build messages in Jinja? Prototypes a Lean-verified Rust agent pipeline.
Sean Welleck retweeted
Amazing progress!
Sean Welleck retweeted
Software Verification in Lean (SVIL) comes to Stanford University on Saturday, November 7.
Co-organized by Beneficial AI Foundation and Lean FRO, it is a day of public lectures on advancing Lean’s software verification ecosystem toward production use, with a focus on end-to-end verification of security-critical systems.
Talks will cover Rust verification with the Aeneas framework, low-level software verification, and formally verified OS kernels.
Speakers include Max Tegmark, Shaowei Lin, Leonardo de Moura, Jonathan Protzenko, Andres Erbsen, and Mike Dodds.
In-person space is limited. Non-local and waitlisted registrants will receive a livestream link, and recordings will be available afterward.
Program and registration: beneficial-ai-foundation.git…
Sean Welleck retweeted
🚀 Introducing LLM2Cert: the first end-to-end framework bringing Lean into agentic synthesis + verification of Lyapunov/barrier certificates, co-led by @Zhipeng0727 and me.
🏆 Automatically synthesized & formally verified safety/stability certificates for 81 of 105 benchmark dynamical systems (77%). On a common subset, 2.1× the successes of 5 prior pipelines combined.
More details in the thread below 🧵👇
Sean Welleck retweeted
That’s a great benchmark to look at
1/5
Verification has become the bottleneck as LLMs start proving conjectures at scale! No worries, Lean can help. But how well can AI agents solve open problems formalized in Lean, end to end?
Introducing reuF2F: 244 open math problems from Research Experiences for Undergraduates (REU) programs, spanning a broad range of math areas, formalized by experts in Lean.
We also build Leanimum-agent, a minimal coding agent for Lean, so every model is compared under the same harness.
Even the best-performing model resolves only 109 out of 244 problems. More below 🧵
Sean Welleck retweeted
If you've noticed a distinct dialect emerging in frontier AI models and struggled to understand what they're talking about, you're not crazy!
We turned to psycholinguistics to investigate what makes the writing style so weird, and why this might be happening
🚨New preprint! 🧵
Sean Welleck retweeted
AI agents can't yet carry out open scientific research
Into the Unknown benchmarks 9 frontier agents on 15 unsolved research directions curated by DOE scientists. None of the 135 submissions meets the scientist's bar, and 114 fail an integrity check for fabricated or inconsistent results.
Sean Welleck retweeted
REUF2F: Benchmarking coding agents on formalized undergraduate research problems. ~ Zeyu Zheng et al. zeyu-zheng.github.io/reuF2F.…
#AI4Math
Sean Welleck retweeted
Happy to see mini-Lean-agent style harnesses becoming popular!
Systems that require complex lean search tools and other scaffolding might be overkill nowadays.
Modern models can perform very well with just access to bash!
Also check out mini-Lean-agent, another recent minimal agent harness for Lean by @tomas__ortega et al!
arxiv.org/abs/2610.17331
New benchmark: reuF2F
244 open math problems from Research Experiences for Undergraduates (REU) programs, formalized in Lean.
Agents are given the Lean statement of an open problem, and produce formal proofs that are checked by Lean.
Paper: zeyu-zheng.github.io/reuF2F.… (1/n)
1/5
Verification has become the bottleneck as LLMs start proving conjectures at scale! No worries, Lean can help. But how well can AI agents solve open problems formalized in Lean, end to end?
Introducing reuF2F: 244 open math problems from Research Experiences for Undergraduates (REU) programs, spanning a broad range of math areas, formalized by experts in Lean.
We also build Leanimum-agent, a minimal coding agent for Lean, so every model is compared under the same harness.
Even the best-performing model resolves only 109 out of 244 problems. More below 🧵
Amazing work and collaborative effort led by Zeyu Zheng (@regunivers)!
Paper: zeyu-zheng.github.io/reuF2F.…
Benchmark: github.com/cmu-l3/reuF2F
Leanimum Agent: github.com/cmu-l3/Leanimum-a… (7/7)
Also check out mini-Lean-agent, another recent minimal agent harness for Lean by @tomas__ortega et al!
arxiv.org/abs/2610.17331
reuF2F is now on arxiv 🎉
arxiv.org/abs/2610.27046
New benchmark: reuF2F
244 open math problems from Research Experiences for Undergraduates (REU) programs, formalized in Lean.
Agents are given the Lean statement of an open problem, and produce formal proofs that are checked by Lean.
Paper: zeyu-zheng.github.io/reuF2F.… (1/n)
Sean Welleck retweeted
Now it's on arXiv!
arxiv.org/abs/2610.27046
1/5
Verification has become the bottleneck as LLMs start proving conjectures at scale! No worries, Lean can help. But how well can AI agents solve open problems formalized in Lean, end to end?
Introducing reuF2F: 244 open math problems from Research Experiences for Undergraduates (REU) programs, spanning a broad range of math areas, formalized by experts in Lean.
We also build Leanimum-agent, a minimal coding agent for Lean, so every model is compared under the same harness.
Even the best-performing model resolves only 109 out of 244 problems. More below 🧵
Sean Welleck retweeted
What is TMLR doing under a 300% increase in submissions?
- Suspensions of >150 authors
- Publicly naming major violations
- Submission quotas distinguishing slop patterns
- Desk rejections 6% → 53%; focus on clarity
- AI reviews evaluating soundness
blog.tmlr.org/2026/adapting-…
