Building the future workforce of the AI economy 🤝
United States
Joined March 2014
- Tweets3.2K
- Following351
- Followers12.1K
- Likes3.9K
ATLAS-Finance is featured in @airstreet’s 2026 State of AI Report. A frontier model scores 100% on structured accounting tasks but only 12.3% on our 100 banking assignments, where passing means a correct workbook delivered the way a bank would expect. @nathanbenaich’s full report: stateof.ai/2026-explore/busi…
Claude scored 100% on structured accounting tasks, but succeeded in only 12% of full banking assignments. Being good at individual financial tasks is different from delivering actual financial work inside an enterprise.
@airstreet's 2026 State of AI Report highlights this gap using our ATLAS-Finance benchmark.
In 2026, AI delivered outsized value to software engineers and vibe coders. 2027 will be a breakout year for financial agents, driven by high-fidelity evaluations like ATLAS-Finance and RL environments that train agents to accurately produce high-value deliverables.
Meta’s Muse Spark 1.3 Evaluation Report describes the human red teaming Handshake’s team did before Muse launched (section 3.1.4.2). Pre-release access, jailbreaks found through the product, full trajectories and analysis sent back, mitigations improved before release.
Report: research.meta.ai/static/muse…
Exciting to see @joinHandshake alongside some of the teams advancing the next generation of AI.
Proud to be part of @nvidia's Nemotron ecosystem!
ChatGPT for Education analyzed thousands of professional chats with AI. After removing routine tasks like email and calendaring, a little over 40% of what remained was people asking questions and doing work outside their role.
Roles are blurring. That makes skills-based and project-based hiring more important than ever: people are stretching beyond their job descriptions, and employers increasingly want that flexibility too.
Handshake works with thousands of working scientists to measure where frontier AI falls short on real research tasks, and to build the data that closes the gap.
Congratulations to @reflection_ai on Beam, released today with a full safety report.
Handshake is named in the report as a source for the conversations Beam was tested against, and among the partners who tested the model before release.
Safety testing is expert work. Before a model ships it has to face the requests that matter: self-harm, child safety, cyber offense, fraud. The hard cases are not the obvious ones. A nurse or a researcher can ask something that sounds dangerous and isn’t. Someone has to write that case and decide where the safe answer ends.
Our people wrote those cases for Beam and judged every turn. Making AI safe takes the same expert judgment as making it useful, and we do both.
Report: reflection.ai/beam-safety-re…
Beam Safety Technical Report — Reflection
Make intelligence open and accessible to all
reflection.ai
Beam is now available across platforms, with open weights under the Apache 2.0 license - in collaboration with an ecosystem of 40+ partners.
We're releasing:
- Weights, tokenizer, and reference implementation
- BF16, FP8 and NVFP4 weights and a DFlash2 drafter
- The open source harness we built and trained Beam on: Mirror CLI
Weights and model card on Hugging Face: huggingface.co/reflection
We collaborated with @NVIDIA AI to post-train Nemotron 3.5 Super VL on over 2,000 data points from experts on top enterprise R&D teams, doubling its accuracy on life sciences artifact interpretation.
@joinHandshake is building accurate, efficient models to let researchers accelerate their work and focus on the scientific decisions that drive discovery.
Read the research report: joinhandshake.com/research/a…
Fellows: your dashboard is changing so you can see what you earned, how it was calculated, and when it lands. You’ll see base pay, incentives, and adjustments by project, plus a reason for every change and payment status.
Rolling out across projects over the next few weeks.
Full update here: joinhandshake.com/blog/our-t…
What a week at @COLM_conf in SF! We loved meeting the research community, hearing about the work people are doing, and sharing more about our ATLAS benchmark collection at the Handshake Al booth.
Also, a huge thank you to our friends at @Cerebras for cohosting Cafe Compute with us. It was a blast!
Explore the full set of ATLAS benchmarks → joinhandshake.com/research/b…
“The model’s capabilities are pretty spiky.” — Jonas Mueller
The best GRE prep used to be a budget question. This benchmark says it doesn't have to be.
@joinHandshake ran a controlled study with 2800+ students comparing the impact of AI with human tutor. Two interesting findings: (1) AI and human produced similar learning gains, and (2) AI is 70x (Opus) to ~900x (Gemma) cheaper. See: ATLAS-Education joinhandshake.com/research/b…
Agent benchmarks should be grounded in how real people actually interact. This often requires user simulation, but current models often lack fidelity.
After months of development, our research team has published CUE, a simulator that outperforms state-of-the-art approaches on fidelity and its ability to reproduce outcomes with real users.
We’ve released the research, code, and trained embedding model openly for researchers to build better multi-turn benchmarks!
Handshake retweeted
A user simulator isn’t useful just because it looks like a real user. AI agents should also succeed/fail with simulated users the same way they do with real ones.
To fix this, we open-sourced CUE, a new simulator to help you build more ecologically valid multi-turn benchmarks.