Yuzhe Yang retweeted
Our agent scientist, Agent S4, just got an open 27B model to beat GPT-5.5 and Claude Opus 4.8 on OSWorld V2, with the model's weights left exactly as they were.
With the interfaces S4 evolved, Qwen3.8-27B completes 21.3% of the 108 tasks, the highest binary success of any open-source computer-use agent and 3.8x the same model with fixed interfaces.
It costs $1.10 per task, against $25.50 for GPT-5.5 and $72.40 for Opus 4.8 ๐ธ
Full write-up: simular.ai/articles/agent-s4โฆ
Check out our latest work on computer-use agents, Agent S4! We believe the future of CUA is not just stronger models, but agents that build and evolve their own interfaces. By learning from experience and improving how they interact with computers, agents can become more capable and efficient. Excited to share this fully open-source work!
Huge thanks to @saa1605 , @chalo2000, @Bhuvan_Koduru, @Richard_simular, JC, @angli_ai, @xwang_lk, and the @SimularAI team for making this happen! Itโs been a pleasure working together and bringing this idea to life.
People long believed in human-like computer use. But software (even terminals) is built for humans, not AI agents. Why not let agents build their own?
Meet ๐๐ด๐ฒ๐ป๐ ๐ฆ๐ฐ, an agent scientist evolving its own interfaces, letting an open 27B model beats GPT-5.5 & Opus 4.8.
With its evolved interfaces, the same model improves ๐ฏ.๐ดร and costs <๐ฑ% as much per task as either frontier model.
Agents today read screenshots and act through click, scroll, type, maybe a shell. So an agent fills a spreadsheet cell by cell when one line of code would do, or prints two thousand lines of a file to find one function.
Every new app or model release means a human reads trajectories, diagnoses failures, and hand-writes new primitives. Skill-distillation methods can reorder what the interface allows, but they canโt extend it. Agents compensate with more test-time compute instead of a better interface.
๐ ๐ผ๐ฑ๐ฒ๐น๐ ๐ถ๐บ๐ฝ๐ฟ๐ผ๐๐ฒ; ๐ถ๐ป๐๐ฒ๐ฟ๐ณ๐ฎ๐ฐ๐ฒ๐ ๐๐๐ฎ๐ ๐ณ๐ฟ๐ผ๐๐ฒ๐ป.
โWe want AI agents that can discover like we can, not which contain what we have discovered.โ
โ Rich Sutton, The Bitter Lesson
Hand-designed interfaces are what we have discovered. Agents can read their own trajectories, spot whatโs missing, and write and test code. ๐๐ป๐๐ฒ๐ฟ๐ณ๐ฎ๐ฐ๐ฒ ๐ฑ๐ฒ๐๐ถ๐ด๐ป ๐๐ต๐ผ๐๐น๐ฑ ๐ฏ๐ฒ ๐ฎ๐ป ๐ฎ๐ด๐ฒ๐ป๐ ๐ฐ๐ฎ๐ฝ๐ฎ๐ฏ๐ถ๐น๐ถ๐๐.
Thatโs what Agent S4 does.
Agent S4 makes a ๐ฟ๐ฒ๐ฝ๐ผ๐๐ถ๐๐ผ๐ฟ๐ ๐ผ๐ณ ๐ถ๐ป๐๐ฒ๐ฟ๐ณ๐ฎ๐ฐ๐ฒ๐ ๐ฎ๐ป๐ฑ ๐๐๐ฎ๐ด๐ฒ ๐ธ๐ป๐ผ๐๐น๐ฒ๐ฑ๐ด๐ฒ the unit of evolution. A task agent attempts synthetic tasks with no success labels or verifiers. A meta-agent studies the trajectories, forms hypotheses about which actions, observations, or knowledge would help, and builds descendants through mutation or crossover. Validation tasks the meta-agent never sees decide which repositories survive.
The model and harness stay fixed, and the meta-agent ๐ป๐ฒ๐๐ฒ๐ฟ ๐๐ฒ๐ฒ๐ ๐ฎ ๐ฏ๐ฒ๐ป๐ฐ๐ต๐บ๐ฎ๐ฟ๐ธ ๐๐ฎ๐๐ธ. ๐ข๐ป๐น๐ ๐๐ต๐ฒ ๐ถ๐ป๐๐ฒ๐ฟ๐ณ๐ฎ๐ฐ๐ฒ ๐ฒ๐๐ผ๐น๐๐ฒ๐.
With the evolved repository, Qwen3.8-27B goes from ๐ฑ.๐ฒ% to ๐ฎ๐ญ.๐ฏ% on OSWorld V2, ๐ฏ.๐ดร the same model with fixed interfaces and a new state of the art for open-source computer-use agents.
It beats GPT-5.5 (13.0%, $25.50/task) and edges past Opus 4.8 (20.6%, $72.40/task) at $๐ญ.๐ญ๐ฌ per task.
On ScienceBoard, the same model goes from 41.4% to 58.6%.
The analysis is the more interesting part. Evolution mostly deepens interfaces the agent already has: 4 new interfaces and 127 extensions. Give the meta-agent a scratch VM to test its code, and it builds 12 new interfaces and 253 extensions, reaching music scores, audio projects, and 3D scenes. The best repository came from crossing two earlier branches.
In one case study, the seed agent hand-builds a browser client mid-task and debugs through failure after failure; the evolved agent calls page_text and reaches the same message at tool call ๐ญ๐ญ instead of ๐ฏ๐ณ.
Human-like computer use was a starting point, not the destination. The interface an agent acts through is a second axis of improvement, beyond better models and harnesses, and agents can now design it themselves. ๐๐๐ผ๐น๐๐ถ๐ป๐ด ๐ฎ๐ด๐ฒ๐ป๐ ๐ถ๐ป๐๐ฒ๐ฟ๐ณ๐ฎ๐ฐ๐ฒ๐ ๐ถ๐ ๐ฎ ๐ป๐ฒ๐ ๐ฑ๐ถ๐บ๐ฒ๐ป๐๐ถ๐ผ๐ป ๐ผ๐ณ ๐ฎ๐ด๐ฒ๐ป๐ ๐๐ฒ๐น๐ณ-๐ถ๐บ๐ฝ๐ฟ๐ผ๐๐ฒ๐บ๐ฒ๐ป๐.
Yuzhe Yang retweeted
People long believed in human-like computer use. But software (even terminals) is built for humans, not AI agents. Why not let agents build their own?
Meet ๐๐ด๐ฒ๐ป๐ ๐ฆ๐ฐ, an agent scientist evolving its own interfaces, letting an open 27B model beats GPT-5.5 & Opus 4.8.
With its evolved interfaces, the same model improves ๐ฏ.๐ดร and costs <๐ฑ% as much per task as either frontier model.
Agents today read screenshots and act through click, scroll, type, maybe a shell. So an agent fills a spreadsheet cell by cell when one line of code would do, or prints two thousand lines of a file to find one function.
Every new app or model release means a human reads trajectories, diagnoses failures, and hand-writes new primitives. Skill-distillation methods can reorder what the interface allows, but they canโt extend it. Agents compensate with more test-time compute instead of a better interface.
๐ ๐ผ๐ฑ๐ฒ๐น๐ ๐ถ๐บ๐ฝ๐ฟ๐ผ๐๐ฒ; ๐ถ๐ป๐๐ฒ๐ฟ๐ณ๐ฎ๐ฐ๐ฒ๐ ๐๐๐ฎ๐ ๐ณ๐ฟ๐ผ๐๐ฒ๐ป.
โWe want AI agents that can discover like we can, not which contain what we have discovered.โ
โ Rich Sutton, The Bitter Lesson
Hand-designed interfaces are what we have discovered. Agents can read their own trajectories, spot whatโs missing, and write and test code. ๐๐ป๐๐ฒ๐ฟ๐ณ๐ฎ๐ฐ๐ฒ ๐ฑ๐ฒ๐๐ถ๐ด๐ป ๐๐ต๐ผ๐๐น๐ฑ ๐ฏ๐ฒ ๐ฎ๐ป ๐ฎ๐ด๐ฒ๐ป๐ ๐ฐ๐ฎ๐ฝ๐ฎ๐ฏ๐ถ๐น๐ถ๐๐.
Thatโs what Agent S4 does.
Agent S4 makes a ๐ฟ๐ฒ๐ฝ๐ผ๐๐ถ๐๐ผ๐ฟ๐ ๐ผ๐ณ ๐ถ๐ป๐๐ฒ๐ฟ๐ณ๐ฎ๐ฐ๐ฒ๐ ๐ฎ๐ป๐ฑ ๐๐๐ฎ๐ด๐ฒ ๐ธ๐ป๐ผ๐๐น๐ฒ๐ฑ๐ด๐ฒ the unit of evolution. A task agent attempts synthetic tasks with no success labels or verifiers. A meta-agent studies the trajectories, forms hypotheses about which actions, observations, or knowledge would help, and builds descendants through mutation or crossover. Validation tasks the meta-agent never sees decide which repositories survive.
The model and harness stay fixed, and the meta-agent ๐ป๐ฒ๐๐ฒ๐ฟ ๐๐ฒ๐ฒ๐ ๐ฎ ๐ฏ๐ฒ๐ป๐ฐ๐ต๐บ๐ฎ๐ฟ๐ธ ๐๐ฎ๐๐ธ. ๐ข๐ป๐น๐ ๐๐ต๐ฒ ๐ถ๐ป๐๐ฒ๐ฟ๐ณ๐ฎ๐ฐ๐ฒ ๐ฒ๐๐ผ๐น๐๐ฒ๐.
With the evolved repository, Qwen3.8-27B goes from ๐ฑ.๐ฒ% to ๐ฎ๐ญ.๐ฏ% on OSWorld V2, ๐ฏ.๐ดร the same model with fixed interfaces and a new state of the art for open-source computer-use agents.
It beats GPT-5.5 (13.0%, $25.50/task) and edges past Opus 4.8 (20.6%, $72.40/task) at $๐ญ.๐ญ๐ฌ per task.
On ScienceBoard, the same model goes from 41.4% to 58.6%.
The analysis is the more interesting part. Evolution mostly deepens interfaces the agent already has: 4 new interfaces and 127 extensions. Give the meta-agent a scratch VM to test its code, and it builds 12 new interfaces and 253 extensions, reaching music scores, audio projects, and 3D scenes. The best repository came from crossing two earlier branches.
In one case study, the seed agent hand-builds a browser client mid-task and debugs through failure after failure; the evolved agent calls page_text and reaches the same message at tool call ๐ญ๐ญ instead of ๐ฏ๐ณ.
Human-like computer use was a starting point, not the destination. The interface an agent acts through is a second axis of improvement, beyond better models and harnesses, and agents can now design it themselves. ๐๐๐ผ๐น๐๐ถ๐ป๐ด ๐ฎ๐ด๐ฒ๐ป๐ ๐ถ๐ป๐๐ฒ๐ฟ๐ณ๐ฎ๐ฐ๐ฒ๐ ๐ถ๐ ๐ฎ ๐ป๐ฒ๐ ๐ฑ๐ถ๐บ๐ฒ๐ป๐๐ถ๐ผ๐ป ๐ผ๐ณ ๐ฎ๐ด๐ฒ๐ป๐ ๐๐ฒ๐น๐ณ-๐ถ๐บ๐ฝ๐ฟ๐ผ๐๐ฒ๐บ๐ฒ๐ป๐.
Yuzhe Yang retweeted
๐ธ For computer-use agents, using a frontier model at every step is expensiveโbut relying on a small model alone is risky.
LEASH offers a middle ground: let a lightweight agent handle what it can, bring in a stronger model only when needed, and keep monitoring afterward. Put your frontier model on a LEASH !! ๐ฆฎ
๐ arxiv.org/pdf/2610.22280
Yuzhe Yang retweeted
ไปๅนดNeurIPS position trackๆไปฌๆไธ็ฏ่ฎบๆ๏ผไธไธชๅฎก็จฟไบบๆไบ9ใ9ใ7๏ผๅนณๅ8.33ใAC็meta reviewๅ็ๆฏ"The reviews are pretty positive"๏ผrebuttalไนๅไธไธชไบบ้ฝ่ฏด็ปดๆๆญฃๅใ
็ถๅPCไธๅฅ่ฏๆไบ๏ผ
"many statements in the paper are superficial and not presented scientifically"
่ฎบๆ่ฎฒ็ๆฏไปไนๅขใAI้กถไผ็ฐๅจๆไธไปถไบๆๅจไธ่ตท๏ผๅฎก็จฟใๅ่กจไผ ๆญใ็คพๅบไบคๆต๏ผๅ
จ้จๅๅจไธๅนดๅ ไธชๅบๅฎๅจๆซ็่ถ
ๅคงไผ่ฎฎไธใไบบๅฐ็ๆถๅ่ฟๅพ้ซๆใ็ฐๅจAIๆนๅๅนณๅๆฏไธชfacultyไธๅนด็้กถไผ่ฎบๆ๏ผ2015ๅนดไธๅฐ2็ฏ๏ผ2024ๅนดๆฅ่ฟ5็ฏใไธๆฌก้ฟ้ๅผไผ็็ขณๆๆพ๏ผๅฏ่ฝ่ถ
่ฟไธไธชไบบไธๅนด็้ขๅบฆใๅฎก็จฟๅๅใๅบ้ฆๅฎน้ใ็ญพ่ฏ๏ผๅ
จ้จไธ่ตท็ใๆไปฌ็ไธปๅผ ๆฏๆ่ฟไธไปถไบๆๅผ๏ผๅ
่ฏ็นๅ่ฏดใ
ๅฅฝ็ฌ็ๅฐๆนๅจ่ฟ้ใไธไธชๅฎก็จฟไบบๅ AC้ฝ่ฏด่ก๏ผๆๅๅ ไธชPCไธๅฅ่ฏๅฐฑ็ฟปไบใไธ็ฏๅๆงฝ"ไธๅนดไธๆฌกใๅ ไธชไบบๆๆฟ"็่ฎบๆ๏ผๆๅไนๆฏไธๅนดไธๆฌกใๅ ไธชไบบๆ็ๆฟใ
่ฟๆไธชๆๆฒๆดๅผๅพ่ฏดใ6ๆ่ฟ็ฏ่ฎบๆๅ
่ขซflag็ไผผAI็ๆ๏ผ่ฆๆไปฌไธคๅจๅ
่ฏๆ่ชๅทฑๆฏไบบใ่ฏๆๆนๅผๆฏไบคๅธฆ็ๆฌๅๅฒ็ๅจ็บฟ็จฟ๏ผๆ ๅบpre-AI็ๆฌใpost-AI็ๆฌๅๆ็ป็๏ผ่ฟ่ฆ้ๆฎต่ฏดๆAIๆนๅจๆฒกๆๅผๅ
ฅๆฐ่ง็นใๆไปฌไบคไบOverleaf็ๅๅฒ๏ผPC็ๅฎๆคๅไบdesk rejectใ
ๆไปฅ็ป่ฟๅจๅ่ฎบๆ็ๆๅไธไธชๅพๅฎ้
็ๅปบ่ฎฎ๏ผไป็ฌฌไธๅคฉ่ตทๅฐฑๅจๆ็ๆฌๅๅฒ็ๅฐๆนๅ๏ผ็จAIๆถฆ่ฒไนๅๆๅจๅญไธไธชcheckpointใ"ไฝ ๆฏไธๆฏไบบ"่ฟไธช้ฎ้ข๏ผไปฅๅไผ่ถๆฅ่ถๅธธ่ขซ้ฎๅฐใ
่ฏดๅฅๅ
ฌ้่ฏ๏ผPCๆๆๅๆ็ปๅคๆญ๏ผposition paperๆไนๆ็ฎscientificๆฌๆฅไนๆฒกๆ็ปไธๆ ๅใๅฐฑๆฏๅๅๆๅจ่ฟ็ฏ่ฎบๆๅคดไธ๏ผๆ็นๅญ็ฌไธๅพใ
๐ Position: AI Conferences Are Under Strain, Unbundling as a Testable Reform
1/6 ๐ง ๐ฌ๐ผ๐๐ฟ ๐๐ ๐๐๐๐ถ๐๐๐ฎ๐ป๐ ๐๐ฎ๐ป ๐ฅ๐ฒ๐บ๐ฒ๐บ๐ฏ๐ฒ๐ฟ ๐ช๐ต๐ฎ๐ ๐ฌ๐ผ๐ ๐ฆ๐ฎ๐ถ๐ฑโ๐ฎ๐ป๐ฑ ๐ฆ๐๐ถ๐น๐น ๐๐ด๐ป๐ผ๐ฟ๐ฒ ๐ช๐ต๐ฎ๐ ๐ฌ๐ผ๐ ๐ช๐ฎ๐ป๐.
Long-running assistants accumulate facts, preferences, and constraints across conversations. But remembering them is not enough: a response can sound fluent and relevant while still recommending exactly what the user has ruled out.
We present ๐ ๐ฒ๐บ๐๐ผ๐น๐ฑ: ๐๐ฒ๐ฎ๐ฟ๐ป๐ถ๐ป๐ด ๐๐ผ๐บ๐ฝ๐ฎ๐ฐ๐ ๐ฆ๐ผ๐ณ๐ ๐ ๐ฒ๐บ๐ผ๐ฟ๐ ๐ณ๐ผ๐ฟ ๐๐ผ๐ป๐ด-๐๐ผ๐ป๐๐ฒ๐
๐ ๐ฃ๐ฒ๐ฟ๐๐ผ๐ป๐ฎ๐น๐ถ๐๐ฎ๐๐ถ๐ผ๐ป ๐๐ถ๐ฎ ๐ข๐ป-๐ฃ๐ผ๐น๐ถ๐ฐ๐ ๐ข๐ฝ๐๐ถ๐บ๐ถ๐๐ฎ๐๐ถ๐ผ๐ป. MemFold trains a model to use compact soft memory by learning from the answers it produces, combining task-level rewards with more detailed token-level guidance.
๐ข๐๐ฟ ๐ธ๐ฒ๐ ๐ณ๐ถ๐ป๐ฑ๐ถ๐ป๐ด๐ ๐ฅ:
๐ฏ ๐๐๐ฑ๐ด๐ฒ ๐บ๐ฒ๐บ๐ผ๐ฟ๐ ๐ฏ๐ ๐๐ต๐ฒ ๐ฎ๐ป๐๐๐ฒ๐ฟ๐ ๐ถ๐ ๐๐๐ฝ๐ฝ๐ผ๐ฟ๐๐. Reconstruction alone does not tell us whether a model will respect a userโs preferences.
๐ ๐ข๐ป๐ฒ ๐๐ฒ๐ ๐ผ๐ณ ๐ฟ๐ฒ๐๐ฝ๐ผ๐ป๐๐ฒ๐, ๐๐๐ผ ๐น๐ฒ๐ฎ๐ฟ๐ป๐ถ๐ป๐ด ๐๐ถ๐ด๐ป๐ฎ๐น๐. GRPO rewards task outcomes, while token-level guidance provides more detailed feedback on the same generated responses.
๐ ๐๐ฎ๐ฟ๐ด๐ฒ๐ฟ ๐ด๐ฎ๐ถ๐ป๐ ๐๐ถ๐๐ต ๐น๐ผ๐ป๐ด๐ฒ๐ฟ ๐ต๐ถ๐๐๐ผ๐ฟ๐ถ๐ฒ๐. Across three Qwen backbones, the margin over the strongest baseline widens from PersonaMem-32K to PersonaMem-128K.
๐ ๐ง๐ฟ๐ฎ๐ป๐๐ณ๐ฒ๐ฟ ๐ฏ๐ฒ๐๐ผ๐ป๐ฑ ๐๐ต๐ฒ ๐๐ฟ๐ฎ๐ถ๐ป๐ถ๐ป๐ด ๐ฑ๐ฎ๐๐ฎ. MemFold transfers to PrefEval and LongMemEval without target-domain training.
๐ ๐๐ต๐ฒ๐ฐ๐ธ ๐ผ๐๐ ๐๐ต๐ฒ ๐ฝ๐ฎ๐ฝ๐ฒ๐ฟ, ๐ฝ๐ฟ๐ผ๐ท๐ฒ๐ฐ๐, ๐ฎ๐ป๐ฑ ๐ฐ๐ผ๐ฑ๐ฒ:
๐ Paper: huggingface.co/papers/2609.3โฆ
๐ Project: memfold.github.io
๐ป Code: github.com/Johnny221B/memfolโฆ
5/6 ๐ ๐ช๐ต๐ฎ๐ ๐๐ฐ๐๐๐ฎ๐น๐น๐ ๐๐ฟ๐ถ๐๐ฒ๐ ๐๐ต๐ฒ ๐๐ฎ๐ถ๐ป๐?
The ablations show that most of the task improvement comes from ๐๐ฅ๐ฃ๐ข: directly rewarding better answers.
๐ง๐ผ๐ธ๐ฒ๐ป-๐น๐ฒ๐๐ฒ๐น ๐ด๐๐ถ๐ฑ๐ฎ๐ป๐ฐ๐ฒ adds a smaller but consistent boost, helping the model make better use of its compact memory.
1
1
65
6/6 ๐ฏ Personalization shows up in the details.
In this example, the user ๐ฑ๐ถ๐๐น๐ถ๐ธ๐ฒ๐ ๐๐ฒ๐ฎ๐ฟ๐ฎ๐ฏ๐น๐ฒ ๐๐ฒ๐ฐ๐ต๐ป๐ผ๐น๐ผ๐ด๐โyet other methods still recommend fitness trackers.
MemFold instead suggests a journal, body measurements, and progress photos. ๐ ๐ฝ๐ฟ๐ฒ๐ณ๐ฒ๐ฟ๐ฒ๐ป๐ฐ๐ฒ ๐ผ๐ป๐น๐ ๐บ๐ฎ๐๐๐ฒ๐ฟ๐ ๐ถ๐ณ ๐๐ต๐ฒ ๐ณ๐ถ๐ป๐ฎ๐น ๐ฎ๐ป๐๐๐ฒ๐ฟ ๐ฎ๐ฐ๐๐๐ฎ๐น๐น๐ ๐ฟ๐ฒ๐๐ฝ๐ฒ๐ฐ๐๐ ๐ถ๐.
1
60
Yuzhe Yang retweeted
Our computer-use agent @sai_borg just beat Opus 5 and GPT-5.6 Sol on OSWorld 2.0.
Sai scored 73% on the CUA benchmark where each complex task takes a skilled human over an hour to execute.
And Sai did it at about 2/3 the cost per task of Opus and GPT ๐ธ
The unlock is our neurosymbolic framework that pairs the exploratory power of neural networks with the logic of symbolic code, resulting in reliability and cost saving.
Read the full report: simular.ai/articles/sai-topsโฆ
Yuzhe Yang retweeted
๐๐ก๐ ๐๐ข๐ญ๐ญ๐๐ซ ๐๐๐ฌ๐ฌ๐จ๐ง ๐จ๐ ๐๐ ๐๐ง๐ญ๐ข๐ ๐๐๐ฆ๐จ๐ซ๐ฒ: memory should be a derived capability that exists because it makes an agent better at acting over time.
๐๐จ๐ซ๐ฅ๐๐๐๐ฆ๐๐ซ๐๐ง๐ is designed around this principle. Rather than evaluating memory as a storage problem, WorldMemArena evaluates memory through ๐๐๐ญ๐ข๐จ๐งโ๐ฐ๐จ๐ซ๐ฅ๐ ๐ข๐ง๐ญ๐๐ซ๐๐๐ญ๐ข๐จ๐ง, instrumenting the full write โ maintain โ retrieve โ use lifecycle across 400 multimodal, multi-session tasks.
And it exposes the findings that should mark the end of the storage-centric era:
โ Storage โ use. Better memory storage and retrieval do not necessarily produce better task performance. Optimizing the component we designed does not optimize the capability we actually care about.
โ Harness-based memory performs best where memory is hardest. Agents that can write files, reorganize context, create artifacts, and interact with persistent environments adapt most effectively in long-horizon settings. They are costly and unstable today, which is exactly what many Bitter Lesson transitions look like before scaling and learning take over.
The deeper move is in what gets measured. Memory shouldn't get a score; it should be inferred from capability: how much does remembering improve performance over time.
WorldMemArena drags evaluation off the static object and into the actionโworld loop, the only place you can tell whether an agent has developed memory or is just simulating it convincingly.
๐คIt is time to rethink how we evaluate agent memory
๐ As agents become longer horizon and more autonomous, memory is no longer just a module for storing past chats.
๐ ๏ธ It determines how agents track changing worlds, learn from past actions, revise outdated information, and reuse experience for future decisions.
๐ This raises three key questions:
Are human designed write store retrieve memory pipelines still the best choice?
If harnesses such as Codex, Claude Code, and OpenClaw already let agents observe, act, call tools, write files, and reorganize context, can memory also be managed by the harness itself?
Do current evaluations really cover how agent memory is used in realistic settings? Many benchmarks are still text centric or single modal, with limited pressure from screenshots, GUIs, tool feedback, and environment changes.
โ Is final QA accuracy enough?
๐ฅ We present WorldMemArena, a multimodal benchmark for evaluating agent memory through action world interaction.
๐ Key insights:
๐งฉ Memory is a lifecycle, not a static cache.
๐ Better memory storage does not necessarily lead to better final performance.
๐ผ๏ธ Multimodal memory remains a major bottleneck for current systems.
๐ Real agentic trajectories expose the fragility of memory systems.
โ๏ธ Harness-based memory is more flexible, but still costly and unstable.
Yuzhe Yang retweeted
โ ๏ธ Your Agent Harness Can Pass Every Task and Still Be Unsafe.
LLM agents now run inside execution harnesses that dispatch tools, allocate resources, and route messages across components. The harness can return a correct final answer while accessing unauthorized resources, leaking context to the wrong agent, or triggering irreversible side effects along the way.
Evaluating the model's output cannot see any of this.
The unit of safety has shifted. It's the harness.
We present HarnessAudit, a trajectory-level framework for auditing LLM agent harness safety, and uncover the following key insights ๐ฅ:
๐จ Completion โ Safety. Task success and safe execution are fundamentally misaligned.
๐ The harness, not the model, is the unit of safety. Most violations happen mid-trajectory, not at termination.
๐ธ๏ธ Multi-agent collaboration expands the risk surface. Inter-agent communication creates entirely new failure modes.
๐ Resource access dominates violations. Agents rarely call wrong tools โ they call right tools on unauthorized resources.
โก Harness design sets the safety ceiling. Framework choice matters more than model choice for safe deployment.
Yuzhe Yang retweeted
Cool paper on diversity collapse in AI agents.
It's a common issue with all the deployed multi-agent systems.
New paper shows that multi-agent LLM systems converge on near-identical outputs over time, even across different architectures and different starting prompts. They call it diversity collapse. The cause is structural coupling. Shared context, shared task descriptions, and mutual feedback pull everyone toward the same attractor.
They measure it formally with metrics like the Vendi score, and the homogenization is real.
Which means the whole sales pitch for multi-agent on creative tasks (brainstorming, hypothesis generation, ideation) partially falls apart unless you explicitly engineer against it. That means having isolated reasoning phases, decoupled evaluation, and heterogeneous agent designs.
If you're running a multi-agent flow on creative work and you haven't tested for this, there's a real chance you're paying five models to produce one answer in a trench coat.
Paper: arxiv.org/abs/2604.18005
Learn to build effective AI agents in our academy: academy.dair.ai/
Yuzhe Yang retweeted
๐ Three papers are accepted to #ICLR2026! Huge congrats to our students and collaborators!
๐น SAFER: Risk-Constrained Sample-then-Filter in LLMs, led by @Ceeqnn & @YFan_UCSC
๐น Presenting a Paper is an Art, led by @liuchen02938149 & @Toby_Yang_7, in collaboration with @OrbyAI
๐น PhyWorldBench, led by @jinggu4ai (now @xAI), in collaboration with @nvidia
Proud of everyone involvedโamazing work! ๐
Yuzhe Yang retweeted
๐ง Can Multimodal Models Think Like Humans?
Most multimodal models reason in rigid pipelines: either see once, then overthink in text or constantly use external visual tools to re-check. Can reasoning and perception be dynamically interleaved like in the human mind?
๐ค Human: ๐ Look โ ๐ง Think in mind โ ๐ Re-look when confidence is low
๐ค DMLR (ours): ๐ Perception โ ๐ง ย Think in latent space โ ๐ Selective re-percept to maximize token confidence
1๏ธโฃ ๐ซฃย Seeing at Every Step Is Unnecessary. Only a small subset of reasoning steps require visual input.
2๏ธโฃย ๐งญย Confidence as the Compass. Confidence captures the modelโs intrinsic state, reflecting accuracy, reasoning quality, and visual grounding.
3๏ธโฃ ๐ง ย Drafting in the "Mind". DMLR directly optimizes think token in latent space, enabling deeper reasoning without additional generation cost.
4๏ธโฃ ๐ย Dynamic Visual Injection Strategy. DMLR selects and injects only the most relevant visual patches, dynamically updated across iterations.
๐ Read on to explore more analysis and insights! ๐
After a long journey of revisions and resubmissions, our paper โTwinMarket: A Scalable Behavioral and Social Simulation for Financial Marketsโ has finally been accepted to NeurIPS 2025. Iโll be at #neurips2025 in San Diego from Dec 1โ8. Our poster session is on the morning of Dec 5.
Iโd be happy to discuss anything related to social agents, social/market simulations, and financial AI. We are also continuing to explore this direction and expect to share follow-up work in the future.
Link: arxiv.org/pdf/2502.01506
Code: github.com/freedomintelligenโฆ
Website: freedomintelligence.github.iโฆ
Thrilled to share that our paper TwinMarket received the ๐ Best Paper Award at Financial AI Workshop @ ICLR 2025!
Huge thanks to my amazing co-authors, my advisor Dr. Hu @sthuyan , and Prof. Wang @wabyking . Special thanks to the generous support from @RBCBorealis!
#ICLR2025
Yuzhe Yang retweeted
Can LLMs really behave like human investors? How do micro-level behaviors drive macro-level market dynamics?
TwinMarket offers an answer by placing thousands of LLM-driven investors in a realistic stock market environment that incorporates social networks, news, and behavioral biases. This setup lets us watch bubbles, crashes, and herding emerge from individual decisions.
Calibrated on real market data and grounded in behavioral finance, TwinMarket scales to 1,000+ agents, reproduces key stylized market facts (volatility clustering, fat tails, etc.), and reveals how social interaction and cognitive biases jointly drive systemic risk. The work is accepted to NeurIPS 2025 and received the Best Paper Award at the ICLR 2025 Financial AI Workshop.
TwinMarket: A Scalable Behavioral and Social Simulation for Financial Markets
CUHK-Shenzhen, Nanjing University
Paper: arxiv.org/abs/2502.01506
Code: github.com/FreedomIntelligenโฆ
Website: freedomintelligence.github.iโฆ
Our report: mp.weixin.qq.com/s/hxarK4Rxwโฆ
๐ฌ #PapersAccepted by Jiqizhixin
Yuzhe Yang retweeted
๐ ๐๐ซ๐๐ฌ๐๐ง๐ญ๐ข๐ง๐ ๐ ๐ฉ๐๐ฉ๐๐ซ ๐ข๐ฌ ๐๐ง ๐๐ซ๐ญ!๐ค
๐ค Ever felt that most presentation tools lack flexibility and creativity?
๐๐ฆ๐ณ๐ฆ๐ญ๐บ ๐ฆ๐น๐ต๐ณ๐ข๐ค๐ต๐ช๐ฏ๐จ ๐ค๐ฐ๐ฏ๐ต๐ฆ๐ฏ๐ต, ๐ง๐ฐ๐ณ๐ค๐ช๐ฏ๐จ ๐ณ๐ช๐จ๐ช๐ฅ ๐ฅ๐ฆ๐ด๐ช๐จ๐ฏ๐ด, ๐ข๐ฏ๐ฅ ๐ฅ๐ฆ๐ฎ๐ข๐ฏ๐ฅ๐ช๐ฏ๐จ ๐ฎ๐ข๐ฏ๐ถ๐ข๐ญ ๐ต๐ธ๐ฆ๐ข๐ฌ๐ด.
๐๐ฏ๐จ๐๐ซ๐๐ฌ๐๐ง๐ญ changes all of that!
โจ EvoPresent is a self-optimizing framework that unites storytelling, design, and feedback to create effortless, engaging presentation videos. ๐ฅ
๐ก ๐๐๐ฒ ๐๐ข๐ ๐ก๐ฅ๐ข๐ ๐ก๐ญ๐ฌ:
๐ ๐๐ซ๐๐ฌ๐๐๐ฌ๐ญ๐ก, the core multi-task RL model, continuously refines both content and design โ ensuring slides that are impactful and visually captivating.
๐ ๐๐ฏ๐จ๐๐ซ๐๐ฌ๐๐ง๐ญ ๐๐๐ง๐๐ก๐ฆ๐๐ซ๐ค is a comprehensive evaluation suite: 650+ top AI papers & diverse formats to assess content and design, and 2000+ slide pairs for aesthetic scoring, defect correction, and design comparison. ๐ฏ
๐ง ๐๐ช๐จ๐ฉ-๐ฒ๐ถ๐ข๐ญ๐ช๐ต๐บ ๐ง๐ฆ๐ฆ๐ฅ๐ฃ๐ข๐ค๐ฌ ๐ฑ๐ฐ๐ธ๐ฆ๐ณ๐ด ๐ค๐ฐ๐ฏ๐ต๐ช๐ฏ๐ถ๐ฐ๐ถ๐ด ๐ด๐ฆ๐ญ๐ง-๐ช๐ฎ๐ฑ๐ณ๐ฐ๐ท๐ฆ๐ฎ๐ฆ๐ฏ๐ต.
โ๏ธ ๐๐ข๐ญ๐ข๐ฏ๐ค๐ฆ ๐ฃ๐ฆ๐ต๐ธ๐ฆ๐ฆ๐ฏ ๐ค๐ฐ๐ฏ๐ต๐ฆ๐ฏ๐ต & ๐ฅ๐ฆ๐ด๐ช๐จ๐ฏ is the secret to presentation excellence.
๐ ๐๐ฎ๐ฅ๐ญ๐ข-๐ญ๐๐ฌ๐ค ๐๐ training boosts generalization in aesthetic awareness tasks.
๐๐ข๐ฌ๐๐ฅ๐๐ข๐ฆ๐๐ซ: the demo video was completely generated by EvoPresent, no human refinement.