Beibin Li retweeted
Better agents may begin with better teachers.
A capable PhD student can lose days to scattered experiment logs and broken workflows. A good advisor identifies these bottlenecks and builds the support that lets the student focus on scientific judgment.
Our new work brings this idea to AI4AI: a Builder learns from a Target’s execution feedback to design better harnesses. We call these reusable lessons meta-skills. Both models’ weights stay frozen; what improves is how one AI supports another.
Check out our newest paper: Learning Meta-Skills for Agent Harness Design in Test-Time AI4AI
📄 arxiv.org/abs/2609.38143
💻 github.com/qiancheng-apodex/…
Beibin Li retweeted
First entry, first place. 🥇
Apodex took #1 on @ProphetArena's weekly leaderboard. Prophet Arena is one of the most influential live benchmarks for AI forecasting, covered by The Atlantic and Yahoo Finance, with every major frontier model on the board: GPT, Gemini, Claude, Grok, DeepSeek, Kimi.
Apodex answered all 70 questions with a Brier score of 0.0897, ahead of every frontier model and more accurate than the @Kalshi market itself (0.1011). Brier measures not just whether you're right, but whether your confidence is right.
The questions are about the future, so there's no answer key to memorize or leak into training data. Reality grades every forecast after it's made.
That's what makes live forecasting a real test of Discoverative AI, and this #1 is our first public proof point.
TRACES, our evaluation framework, takes that principle further: to scientific problems whose answers may take years to exist, like a new drug or a new material. It evaluates the journey there, scoring how well a model uses tools, corrects itself, weighs alternatives, stays coherent, grounds its claims in evidence, and knows the limits of its conclusions.
Explore TRACES: discovertraces.ai/
#AIForecasting #PredictionMarkets #AIBenchmarks #DiscoverativeAI #Apodex
Beibin Li retweeted
Apodex 1.1 mini is now available in GGUF 🚀
Built for easier local deployment and community experimentation.
GGUF → huggingface.co/apodex/Apodex…
Explore all Apodex 1.1 model formats → huggingface.co/collections/a…
Beibin Li retweeted
GPT-6 Astra @OpenAI is here. Everyone's asking how smart it is.
We asked a different question: can it discover the unknown?
We ran it through TRACES — our benchmark for scientific discovery capability. Six process dimensions, not one score.
Result: it tops our leaderboard. Real generational jump — Evidence and Coherence both up ~20-25% over GPT-5.6 Sol and GPT-5.5. Highest Coherence we've ever recorded.
But the six dimensions don't move together, and that's exactly why one "how smart" number isn't enough. Some capabilities leap forward, others hold steady, in the same model.
That unevenness is what a process score is built to catch.
Full breakdown, live: discovertraces.ai/liveleader…
#GPT6 #TRACES #AIforScience #LLM #AIBenchmark #Apodex #AGI #AIResearch #ScientificDiscovery #FoundationModels
ALT The Model Arena — TRACES capability breakdown across seven frontier models.
Beibin Li retweeted
What does TRACES actually measure? Not merely whether a model reaches the right answer—but whether it can reason its way there.
website: traces.apodex.com/
Six capabilities that spell its name:
*T — Tools: selecting, calling and correctly interpreting external tools
*R — Repair: locating and correcting its own errors once feedback arrives
*A — Alternatives: laying out competing hypotheses and keeping or discarding them as evidence accumulates
*C — Coherence: holding state, constraints and logic intact across a long chain of work
*E — Evidence: grounding every conclusion in observation, data, experiment or citation
*S — Scope: stating the conditions under which a conclusion holds, and where it does not apply
TRACES evaluates today whether the discovery process is rigorous and evidence-grounded, even when the final outcome is not yet known 🔍
Beibin Li retweeted
Most AI benchmarks test whether a model can answer questions with known answers.
TRACES asks a harder question: Can AI make discoveries when the answers are not yet known?
We evaluate models and agents across six capabilities: Tools, Repair, Alternatives, Coherence, Evidence, and Scope.
Explore the leaderboard:
traces.apodex.com/liveleader…
Submit your solver:
traces.apodex.com/submit-you…
#AI #AIAgents #ScientificDiscovery
Beibin Li retweeted
Most AI benchmarks test retrieval — can a model find the known answer? However, the hardest problems in science require discovery, can a system earn an answer nobody has yet?
Meet TRACES 🧭 — the world's first benchmark for measuring discoverative AI: AI that can work through evidence, test hypotheses, and reach verifiable conclusions on problems without answer keys. Proposed by our founder @tianqiao_chen, who defined its six capabilities.
Three things published today: a definition of "discoverative intelligence", a rubric to tell sound investigation from lucky guesses, and a open call for both solvers and problems
*Website: traces.apodex.com/
Beibin Li retweeted
Congrats @Apodex_AI on launching Apodex 1.1🌍
This is an incredibly powerful model for complex agentic tasks. We already saw people sharing use cases like finance analysis and personalized research, and getting very good and faithful results.
We are proud to be on the model card as a deployment solution, and SGLang features like radix cache will make it fast and efficient under asynchronous agent team workflows🫡
Command to run below👇
Meet Apodex 1.1: Scaling Agentic Intelligence for Complex Work
Open Source Harness: github.com/ApodexAI/Frontier…
Open Weights: huggingface.co/collections/a…
We’re excited to introduce Apodex 1.1, our new model family built to scale agentic intelligence for professional work.
🧠 Frontier-level intelligence for complex work
Apodex 1.1 brings frontier-level agentic performance across complex professional work, scientific research, financial analysis, and deep search.
🤝 Asynchronous Agent Team
Apodex 1.1 can break down complex tasks, coordinate multiple agents in parallel, continuously integrate their findings, and let you step in to guide or redirect the work at any time.
🔬 Open-source research workbench
We’re open-sourcing FrontierAgent—a locally deployable research workbench for the Apodex 1.1 family, including asynchronous Agent Team.
Available now:
🔹 Apodex 1.1 — our most capable frontier model, available through the Apodex online workbench
🔹 Apodex 1.1 mini — open-weight model for running complex work locally
🔹 FrontierAgent — open-source, locally deployable research workbench
🌐 Live workbench: apodex.ai/
🦾 API platform: platform.apodex.ai/
📃 Paper: apodex.com/pdf/20260824
Beibin Li retweeted
Meet Apodex 1.1: Scaling Agentic Intelligence for Complex Work
Open Source Harness: github.com/ApodexAI/Frontier…
Open Weights: huggingface.co/collections/a…
We’re excited to introduce Apodex 1.1, our new model family built to scale agentic intelligence for professional work.
🧠 Frontier-level intelligence for complex work
Apodex 1.1 brings frontier-level agentic performance across complex professional work, scientific research, financial analysis, and deep search.
🤝 Asynchronous Agent Team
Apodex 1.1 can break down complex tasks, coordinate multiple agents in parallel, continuously integrate their findings, and let you step in to guide or redirect the work at any time.
🔬 Open-source research workbench
We’re open-sourcing FrontierAgent—a locally deployable research workbench for the Apodex 1.1 family, including asynchronous Agent Team.
Available now:
🔹 Apodex 1.1 — our most capable frontier model, available through the Apodex online workbench
🔹 Apodex 1.1 mini — open-weight model for running complex work locally
🔹 FrontierAgent — open-source, locally deployable research workbench
🌐 Live workbench: apodex.ai/
🦾 API platform: platform.apodex.ai/
📃 Paper: apodex.com/pdf/20260824
Beibin Li retweeted
Today we're launching Miles v0.1, an open-source RL framework for LLMs and multimodal models.
RL training is easy to start and hard to debug. Miles helps you ensure your run is correct, use hardware efficiently, and keep RL running at scale.
Over the past 9 months, 72 contributors have landed 1,326 commits, 85 GPU E2E CI tests, battle-testing Miles on frontier open models like Kimi K3, DeepSeek V4, Qwen 3.8, GLM 5.2, Inkling, MiniMax H3, etc.
Miles powers frontier-model development and production RL workloads at @humansand, @periodiclabs, @modal, @DecagonAI, @Eigent_AI, @nebiusai, @IBM and more, on both @NVIDIAAI and @AIatAMD hardware.
Here is what we built, and why teams picked Miles🧵
Beibin Li retweeted
Our Chief Scientist @SimonShaoleiDu recently explained where single-agent reasoning hits a wall:
Even a one-million-token context may not be enough for one agent to carry a hard research problem.
A new NVIDIA preprint makes that measurable.
Here’s what breaks inside a single context — and how we reorganize the work:
youtu.be/0dqpkTPUd-0?si=9mXc…
Beibin Li retweeted
For my first post, I’m sharing a letter @NVIDIA signed on why open models matter.
AI will transform every industry, power every company, and be built by every country.
Open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty.
The world needs both frontier closed models and frontier open models.
images.nvidia.com/pdf/Open-W…
Beibin Li retweeted
A super long overdue (3+ years?) post on scaling laws.
Compute is expensive. Scaling laws are a way to help us reason about the optimal compute allocation between data and model size before committing to a large run.
The post covers what scaling laws predict, how compute-optimal allocation works, why Kaplan et al. and Chinchilla disagree, and how data limits + fitting details make extrapolation tricky.
lilianweng.github.io/posts/2…
Beibin Li retweeted
@EthanHe_42 worked with me and @imhaotian for half a year on Grok Imagine. I don't think he was being intentional overclaiming here, but the internet narrative quoting him as the "lead" was werid. It was really driven by @imhaotian most of the time.
Good times — especially that intense three-month sprint at the start. Huge credit goes to @imhaotian and several other key people like @zeliu_ @jathushan @ZhibeiM @hexiang @JackCaiXun @chaitu, and later @jia_xuhui @YknZhu, along with many others — especially the latest Grok Imagine 1.5.
In @latentspacepod podcast, I shared my view on video generation, world models, LLMs, agents, continual learning and where the next frontier is.
1. Video models get most of their intelligence from language, not from video data.
2. Idea-to-code is fast now. The bottleneck is back to having enough compute to try every idea.
3. Iteration speed beats almost everything else in model development.
4. The next leap won't be a better video model. It'll be a video agent.
5. Diffusion will be the frontend of AGI, the LLM the backend. Generative UI will replace HTML/CSS: user intent straight to pixels.
6. Physical embodiment may become a tool a powerful AI picks up. Robotics may get solved by video-capable LLMs.
7. Continual learning may look like models that manage their own context, and even rewrite their own harness at test time.
Thanks @swyx and @vibhuuuus for having me 🙏
youtube.com/watch?v=jPtQlILf…
Beibin Li retweeted
xAI just lost the guy who trained Grok.
GPT-4 co-author. Yale PhD. 39,000+ citations.
Gone.
80+ departures in a year.
11 co-founders. 0 remaining.
Musk calls it "rebuilding from scratch."
Employees call it quitting.
#DINQ #xAI
I left xAI earlier this year after two incredible years.
I joined still early in my career, and had the privilege of meeting many great people and friends along the way. Over time, I ended up leading pretraining across architecture, training, and parts of infra, and later led major RL runs for Coding. Together, we pushed forward generation after generation without missing a beat. None of this would have been possible without the fantastic team.
Grateful for my team and friends —their talent and dedication made this possible. Working through hard problems together was the most rewarding part of the journey, and I’ll miss it deeply.
It’s hard to walk away from people you’ve spent years building with. But I’m proud of everything we built together, and ready for the next chapter.
Beibin Li retweeted
Started with one Claude session.
42 days later: 4 of our AI agents cold-review each other's PRs, present at conferences in my place, and heal each other's failures across owners.
Yours not by configuration — yours by accumulation.
github.com/sonichi/sutando
Cool~
xAI has launched Grok 4.3, achieving 53 on the Artificial Analysis Intelligence Index with improved agentic performance, ~40% lower input price, and ~60% lower output price than Grok 4.20
The release of Grok 4.3 places @xAI just above Muse Spark and Claude Sonnet 4.6 on the Intelligence Index, and a 4 points ahead of the latest version of Grok 4.20. Grok 4.3 improves its Artificial Analysis Intelligence Index score while reducing cost to run the benchmark suite.
Key Takeaways:
➤ Grok 4.3 improves on cost-per-intelligence relative to Grok 4.20 0309 v2: it scores higher on the Intelligence Index while costing less to run the full benchmark suite. Grok 4.3 costs $395 to run the Artificial Analysis Intelligence Index, around 20% lower than Grok 4.20 0309 v2, despite using more output tokens. This makes it one of the lower-cost models at its intelligence level
➤ Large increase in real world agentic task performance: The largest single benchmark improvement is on GDPval-AA, where Grok 4.3 scores an ELO of 1500, up 321 points from Grok 4.20 0309 v2’s score of 1179 Grok 4.3, surpassing Gemini 3.1 Pro Preview, Muse Spark, Gpt-5.4 mini (xhigh), and Kimi K2.5. Grok 4.3 narrows the gap to the leading model on GDPval-AA, but still trails GPT-5.5 (xhigh) by 276 Elo points, with an expected win rate of ~17% against GPT-5.5 (xhigh) under the standard Elo formula
➤ Grok 4.3’s performs strongly on instruction following and agentic customer support tasks. It gains 5 points on 𝜏²-Bench Telecom to reach 98%, in line with GLM-5.1. Grok 4.3 maintains an 81% IFBench score from Grok 4.20 0309 v2
➤ Gains 8 points on AA-Omniscience Accuracy, but at the cost of lower AA-Omniscience Non-Hallucination Rate of 8 points, so Grok 4.20 0309 v2 still leads AA-Omniscience Non-Hallucination Rate, followed by MiMo-V2.5-Pro, in line with Grok 4.3
Congratulations to @xAI and @elonmusk on the impressive release!
Beibin Li retweeted
Honored to receive Alumni Achievement Award from UIUC's @siebelschool Siebel School of Computing & Data Science.
Sutando — my personal AI, running on my MacBook — co-presented with me at the @UofIllinois seminar and AIWILD @iclr_conf .
Photo with my advisor Prof. Jiawei Han.
