Official SWE-bench Account. Follow for updates to the SWE-universe
Joined May 2025
- Tweets26
- Following18
- Followers415
- Likes104
More SWE-bench environments, tasks, trajectories, and training recipes for all!
Join us in SWE-bench slack if you're interested in contributing and using these new datasets! (bottom left of swebench.com)
Expect a lot more to come in the following weeks :)
SWE-bench retweeted
New eval! Code duels for LMs ⚔️
Current evals test LMs on *tasks*: "fix this bug," "write a test"
But we code to achieve *goals*: maximize revenue, cut costs, win users
Meet CodeClash: LMs compete via their codebases across multi-round tournaments to achieve high-level goals
SWE-bench retweeted
🏆Glad to know that our #ASE25 paper about automated bug repair using MMLM just got the ACM SIGSOFT Distinguished Paper Award🎉 And it is still ranked top #1 in @SWEbench Mulmimodal Track! Thank Kai, Xiaofei @xfxie312, and Jian for the great work!
Excited to announce Kai's latest ASE'25 work, let LLMs not only see bugs, but also fix them:
📄 “Seeing is Fixing: Cross-Modal Reasoning with Multimodal LLMs for Visual Software Issue Repair”
🔗arxiv.org/abs/2506.16136
Ranked #1 on @SWEbench Multimodal!
SWE-bench retweeted
Congrats to @Zai_org GLM-4.5 on getting the 7th spot on our SWE-bench Verified [Bash Only] leaderboard!
w/ @KLieret @_carlosejimenez @jyangballin
SWE-bench retweeted
Super excited to have @anyscalecompute use mini-swe-agent for their large scale runs!
w/ @KLieret @_carlosejimenez @jyangballin
SWE-bench retweeted
3 out of the top 6 most downloaded datasets on @huggingface are SWE-bench related.
Thanks!!! ♥️
SWE-bench retweeted
What if your agent uses a different LM at every turn? We let mini-SWE-agent randomly switch between GPT-5 and Sonnet 4 and it scored higher on SWE-bench than with either model separately. Read more in the SWE-bench blog 🧵
ALT GPT-5 or Sonnet 4. Flipping a coin at every step of your agent beats both
SWE-bench retweeted
Deepseek v3.1 chat scores 53.8% on SWE-bench verified with mini-SWE-agent. Tends to take more steps to solve problems than others (flattens out after some 125 steps). As a result effective cost is somewhere near GPT-5 mini. Details in 🧵
SWE-bench retweeted
Super exciting to have 3 new open-weight models that all obtain more than 60 on SWE-bench Verified! Looking forward to the results on SWE-bench Multimodal when these models obtain vision capabilities :)
SWE-bench retweeted
Releasing mini, a radically simple SWE-agent: 100 lines of code, 0 special tools, and gets 65% on SWE-bench verified!
Made for benchmarking, fine-tuning, RL, or just for use from your terminal.
It’s open source, simple to hack, and compatible with any LM! Link in 🧵
ALT mini-SWE-agent: 100 lines is all you need. 65% on SWE-bench verified with Claude Sonnet 4. No special tools, no tool calls, no shell session.
🎉 Congrats @Alibaba_Qwen @huybery @JustinLin610 and the Qwen team!
Incredible progress in the last year, love to see Qwen continue championing open models for SWE-bench!
>>> Qwen3-Coder is here! ✅
We’re releasing Qwen3-Coder-480B-A35B-Instruct, our most powerful open agentic code model to date. This 480B-parameter Mixture-of-Experts model (35B active) natively supports 256K context and scales to 1M context with extrapolation. It achieves top-tier performance across multiple agentic coding benchmarks among open models, including SWE-bench-Verified!!! 🚀
Alongside the model, we're also open-sourcing a command-line tool for agentic coding: Qwen Code. Forked from Gemini Code, it includes custom prompts and function call protocols to fully unlock Qwen3-Coder’s capabilities. Qwen3-Coder works seamlessly with the community’s best developer tools. As a foundation model, we hope it can be used anywhere across the digital world — Agentic Coding in the World!
💬 Chat: chat.qwen.ai/
📚 Blog: qwenlm.github.io/blog/qwen3-…
🤗 Model: hf.co/Qwen/Qwen3-Coder-480B-…
🤖 Qwen Code: github.com/QwenLM/qwen-code
