Dynamical Systems (@dynamical__) header
Dynamical Systems (@dynamical__) profile picture

An integrated materials lab to develop and manufacture materials that make new hardware possible.

New York, NY
Joined October 2026
Photo by Dynamical Systems (@dynamical__)Photo by Dynamical Systems (@dynamical__)Photo by Dynamical Systems (@dynamical__)Photo by Dynamical Systems (@dynamical__)Photo by Dynamical Systems (@dynamical__)
Dynamical Systems retweeted
Why do we need more benchmarks for frontier models? What you can evaluate and understand, you can optimize. Excited to share some of our work on connecting scientific reasoning, experiments, measurements and process decisions. @dynamical__ SDL-2 is an open materials science benchmark that follows the whole investigation, from discovering which relationships matter and choosing experiments that distinguish competing explanations to revising an explanation when results surprise and knowing when the evidence is strong enough to act. It asks agents to infer how much results vary from a few observations and choose experiments that identify which uncertainty would change the decision. Code, harness, environments and all traces are open!
Can agents use experimental feedback to establish new material capabilities that hardware teams can reliably design around? Today we’re releasing SDL-2, an open evaluation of how agents choose experiments and decide whether a material process will reliably meet a requirement. We’re interested in the full process of scientific investigation. That means discovering which relationships matter, choosing experiments that distinguish competing explanations, revising an explanation when results surprise and knowing when the evidence is strong enough to act. The experiments run in a digital twin of magnet processing, calibrated on measurements from our campaign with Rensselaer Polytechnic Institute (@rpi). Agents investigate unfamiliar powders with an uncertain physics model, a limited experimental budget and results that vary from batch to batch. Across 32 simulated worlds, frontier models reached correct decisions supported by their own measurements in 44–50% of investigations, compared with 25–28% for other language models and classical experiment design. One in three of their correct answers were unsupported. Our larger ambition is for materials development to keep pace with hardware design, so teams can design around the capabilities they request, not only those that already exist. SDL-2 makes one part of that measurable, how agents investigate. We're releasing the evaluation code, harness, digital twin and all 256 investigation traces. Technical Report: dynamicalsystems.ai/blog/sdl… Code: github.com/Dynamical-Systems… Dataset: huggingface.co/datasets/Dyna…
1
2
8
454