carlos retweeted
I completed my postdoc at Princeton University 9 months ago but never wrote a goodbye post so:
My two years at Princeton were the best 2 years of my life. I'm forever grateful to @karthik_r_n, @prfsanjeevarora & Ellen DiPippo for creating the incredible environment that enabled me to be take part in amazing research.
I'm forever grateful to @closji & @jyangballin for meeting me on my fourth day on the job and then inviting me to join the software engineering benchmark they had developed. And to @KLieret who joined us just after and became a core part of the SWE-team.
Thank you @ShunyuYao12 who met with me when I was making the decision; I enjoyed our walks so much I decided to join just so I could go on more walks with Shunyu.
Thank you @_awettig my collaborator in both science and art, and European<->American linker.
Thank you to my FAIR postdoc hosts @syhw and @sidawxyz.
Thank you to *all* of my collaborators, including first authors: @MinyangTian1 SciCode and CritPt, @AbramovichTalor SWE-agent for Cyber Security, @OriYoran AssistantBench, @18jeffreyma SWE-fficiency, @a1zhang VideoGameBench, @MiniHui_zhu CritPt, and @ori_press my actual brother and author of AlgoTune
Thank you to my senior collaborators: @Diyi_Yang & @lschmidt3 (through John), @haopeng_uiuc (through Minyang), @profvjreddi (through Jeff), @JonathanBerant (through Ori Yoran) and @MatthiasBethge (through Ori Press).
Thank you to all the people at Princeton who made PLI what it is: @danqi_chen @cocosci_lab and all of the other faculty, postdocs, graduate and undergraduate students.
carlos retweeted
Replying to @princeton_nlp
@princeton_nlp graduated 11 (!) PhDs at the hooding ceremony yesterday. Their research has fundamentally shaped the global AI landscape in so many ways, and they have been a core part of building the NLP group and its collegial spirit. It's been a real privilege working with and getting to know them all over the past few years!
@ZexuanZhong @AmeetDeshpande_ @SadhikaMalladi @_carlosejimenez @JensTuyls @VishvakM @__howardchen @xiamengzhou @gaotianyu1350 @danfriedman0 @_awettig
w/ @danqi_chen @prfsanjeevarora
carlos retweeted
A statement on the comments from Secretary of War Pete Hegseth.
anthropic.com/news/statement…
carlos retweeted
Replying to @OfirPress @pewdiepie
@_carlosejimenez the perfectionism that went into the swe llama has finally paid off
carlos retweeted
Updates to official SWE-bench leaderboard: GPT 5.2 high is #3 after Gemini for similar price point. GPT 5.2 medium reduces gap to Sonnet 4.5 and is much cheaper. All independently evaluated using identical mini-swe-agent setup. More info & full trajectories in 🧵
carlos retweeted
Presenting this Thursday Morning at #NeurIPS2025, reach out if you wanna chat and become friend 🙂
Come see the SWE smith poster at Neurips, hall E #113 !
carlos retweeted
Updates to the official SWE-bench leaderboard: Minimax M2 is best open source model (but expensive!). Deepseek v3.2 reasoning close behind, very cheap, but very slow. GLM 4.6 reaches good performance fast and cheap. All independently evaluated with mini-swe-agent. Details in 🧵
carlos retweeted
If you're curious to read how Opus 4.5 solves its tasks, we've just updated the leaderboard on swebench .com including links to full transcripts of the agent
carlos retweeted
Opus 4.5 reclaims the top of the official SWE-bench leaderboard with 74.4%, narrowly ahead of Gemini 3. Cheaper than Opus 4, but more expensive than Gemini. Takes less steps than Sonnet 4.5, but still run for >100 steps for optimal performance. Details in 🧵
carlos retweeted
New eval! Code duels for LMs ⚔️
Current evals test LMs on *tasks*: "fix this bug," "write a test"
But we code to achieve *goals*: maximize revenue, cut costs, win users
Meet CodeClash: LMs compete via their codebases across multi-round tournaments to achieve high-level goals
carlos retweeted
One of my fave papers from COLM by @BenShi34! Really cool eval too (a friends and family Turing test!)
carlos retweeted
We evaluated Anthropic's Sonnet 4.5 with our minimal agent. New record on SWE-bench verified: 70.6%! Same price/token as Sonnet 4, but takes more steps, ending up being more expensive. Cost analysis details & link to full trajectories in 🧵
carlos retweeted
Accepted to #NeurIPS2025!
Big shoutout to our ~120 participants, who graciously allowed me to pester them daily with reminder emails, bug fixes, and troubleshooting queries 😓
carlos retweeted
Deepseek v3.1 chat scores 53.8% on SWE-bench verified with mini-SWE-agent. Tends to take more steps to solve problems than others (flattens out after some 125 steps). As a result effective cost is somewhere near GPT-5 mini. Details in 🧵
carlos retweeted
What if your agent uses a different LM at every turn? We let mini-SWE-agent randomly switch between GPT-5 and Sonnet 4 and it scored higher on SWE-bench than with either model separately. Read more in the SWE-bench blog 🧵




























