@NovaSkyAI

Building SkyRL at @BerkeleySky Join the Slack community: https://xitter.cf/t.co/mSO97T61vR

 Berkeley, California
github.com/NovaSky-AI/SkyRL
Joined January 2025
  • Tweets179
  • Following18
  • Followers2.9K
38Media
We are excited to announce that SkyRL now implements the Tinker API. Run Tinker training scripts on your own hardware with zero code changes. Try it out today: novasky-ai.notion.site/skyrl…

SkyRL Brings Tinker to Your GPUs | Notion

Tyler Griggs, Philipp Moritz, Eric Tang, and the SkyRL Team

novasky-ai.notion.site
Tyler Griggs@tyler_griggs_Feb 13
SkyRL now implements the Tinker API. Now, training scripts written for Tinker can run on your own GPUs with zero code changes using SkyRL's FSDP2, Megatron, and vLLM backends. Blog: novasky-ai.notion.site/skyrl… 🧵
4
27
3,366
NovaSky retweeted
Bryan Lu
@yuzhe_lu6h
Great to see @modal launch this! You can also deploy a Tinker endpoint directly on AWS EC2 with SkyRL @NovaSkyAI for the same cost and flexibility benefits, and start agentic RL training with our AgentCore RL Toolkit! github.com/awslabs/agentcore…
Modal
@modal8h
Tinker changed how people do post-training. We're big fans, and so are many of the teams we work with. But as those teams scale up, they keep raising the same two problems: per-token pricing gets expensive quickly, and you’re limited to the models and features the provider decides to support. We think a good API deserves more than one implementation. Today we're releasing Spindle, an open-source, Tinker-compatible training server that runs in your Modal workspace. Your existing Tinker scripts run unchanged, and because you own the server, you pay for GPU time instead of tokens.
1
3
13
1,433
NovaSky retweeted
Qiuyang Mang
@MangQiuyang13h
FrontierCS 2 is our new benchmark for CS research & development. Each task is based on a paper: we delete the contribution from the released code, and ask agents to outdo the original method, as judged by the authors' chosen metrics. Agents don't have access to the paper or web.
22
50
27
238
32,519
NovaSky retweeted
SkyPilot
@skypilot_org11h
Post-training a 4B model with RL to fly Tesla Roadster. It outperforms recent open models and even GPT-6 Astra in a storm. The key: have the model learn from the teachers first and then learn itself in practice. Trained with SkyRL (@NovaSkyAI) and vLLM (@vllm_project), orchestrated by SkyPilot's Job Group (@skypilot_org). Check it out ⬇️
Zhanghao Wu
@Michaelvll112h
Tesla's flying Roadster went viral. The real demo had been delayed two weeks "due to high winds." We asked: could the latest LLMs sit in the pilot's seat and fly the car through that wind? We built an RL environment for LLMs to fly it. In real time, GPT-6 Astra, DeepSeek, Muse Spark Open, Beam and Nemotron 3.5 Super VL landed 0% of kickflips. Even with unlimited time to think for each decision, they still did worse than a Qwen 4B model we post-trained with RL in a storm.
1
5
14
2,354
NovaSky retweeted
avi basnet
@avigyabasnet13h
Super excited to share some of the work I’ve been doing on supporting 1M token context training on SkyRL for GLM 5.3 Flash, 64x higher than the out of the box Megatron baseline! 🚀 This work was done in collaboration with the amazing folks at @LilaSciences! 🤩 Getting there took key algorithmic and systems optimizations in four main areas: 1) DeepSeek Sparse Attention: run DSA on a sparse kernel; chunk and shard its quadratic indexer. 2) Activation recompute and offloading: recompute activations in backward and offload saved inputs to CPU. 3) Context parallelism: split each sequence across GPUs, with new CP support for KDA and a fix for DSA. 4) Avoiding unnecessary precision upcasts: keep FP32 copies out of memory and off the network. For more details on our implementations and our results, check out our full technical blog here: blog.skyrl.ai/blog/1m-contex…! This work benefited from the contributions of many people along the way. Big thanks to @pcmoritz, @erictang000, @sumanthrh, @CyrusHakha, @richliaw from @anyscalecompute and Keith Truongcao, Tushar Khot, Vivek Subramanian, Semih Kara from Lila Sciences for all the help and guidance!
3
20
5
109
9,022
NovaSky retweeted
Zhanghao Wu
@Michaelvll112h
Check our blog: skypilot.ai/blog/flying-road… Trained with SkyRL (@NovaSkyAI), served with vLLM (@vllm_project), and orchestrated with SkyPilot (@skypilot_org)

Post-training a 4B model with RL to fly the Tesla Roadster 21% better than GPT-6 Astra

We RL-trained Qwen3-4B to fly a simulated flying Roadster. In storms, it beats DeepSeek, Muse, Beam, Nemotron and GPT-6 Astra, even when they get unlimited time to think.

skypilot.ai
5
8
285
NovaSky retweeted
Zhanghao Wu
@Michaelvll112h
Tesla's flying Roadster went viral. The real demo had been delayed two weeks "due to high winds." We asked: could the latest LLMs sit in the pilot's seat and fly the car through that wind? We built an RL environment for LLMs to fly it. In real time, GPT-6 Astra, DeepSeek, Muse Spark Open, Beam and Nemotron 3.5 Super VL landed 0% of kickflips. Even with unlimited time to think for each decision, they still did worse than a Qwen 4B model we post-trained with RL in a storm.
2
8
4
19
4,128
Model customization recipes are live on our website and in the cookbook repo. Beam is now supported across RL and SFT stacks including @axolotl_ai, NeMo RL, NeMo AutoModel, Megatron Bridge (@NVIDIAAI), Miles (@radixark), SkyRL (@NovaSkyAI), and TRL (@huggingface). Get cooking 🧑‍🍳 developers.reflection.ai/cus…

Customize Beam - Reflection Developer Docs

Fine-tune and post-train Beam with the open-source training stacks that support it

developers.reflection.ai
5
9
5
121
7,531
First day 0 model support on SkyRL. Great work from @sumanthrh !
Sumanth Hegde
@sumanthrhOct 16

Day 0 support for Beam on SkyRL

Article · @sumanthrh
3
30
1,205
NovaSky retweeted
vLLM
@vllm_projectOct 16
🎉 Day-0 RL support for Beam in SkyRL @NovaSkyAI. LoRA, BF16 full fine-tuning and MXFP8 RL, with vLLM rollouts and Reflection’s Megatron-Bridge implementation. SkyRL reports +62% LoRA decode throughput from right-sizing expert adapters in vLLM: 276 → 447 tok/s/GPU at 32 concurrent requests, PP2×TP8. This is a decode result, not an end-to-end training speedup. R3 works with pipeline parallelism, and FP32 router weights are preserved during weight sync. Thanks to Sumanth Hegde and the SkyRL team! docs.skyrl.ai
1
6
8
1,086

Day 0 support for Beam on SkyRL

Article · @sumanthrh
13
5
40
4,039
Great to see this out! Datadog's new SOTA model for investigating production incidents. Their in-house training stack builds on top of SkyRL! 🚀🚀🚀
Datadog, Inc.
@datadoghqOct 15
Meet Scout 1.0, our first AI model built for investigating production incidents. Scout matches frontier-model investigation quality at a fraction of the cost, which makes it practical to investigate more issues, earlier. We trained Scout on realistic simulations built from real-world incidents and production telemetry, using reinforcement learning to reward accurate root causes backed by evidence. Learn more and sign up for the preview: bit.ly/3TJknGO
2
6
24
1,251
NovaSky retweeted
Eric Tang@erictang000Oct 15
datadog ai research used SkyRL to post-train GLM 5.3 Flash to perform at the pareto frontier for their SRE eval @bthecohen and team were some of the earliest users of SkyRL at scale, really cool to see these results finally out! great tech report as well
Datadog, Inc.
@datadoghqOct 15
Meet Scout 1.0, our first AI model built for investigating production incidents. Scout matches frontier-model investigation quality at a fraction of the cost, which makes it practical to investigate more issues, earlier. We trained Scout on realistic simulations built from real-world incidents and production telemetry, using reinforcement learning to reward accurate root causes backed by evidence. Learn more and sign up for the preview: bit.ly/3TJknGO
2
6
36
1,235
Datadog post-trained GLM-5.3 Flash via RL on real-incident sims. Matches frontier SRE performance at much lower cost. Their RL stack is built on top of SkyRL! Check out the technical report web-assets.dd-static.net/425…
Datadog, Inc.
@datadoghqOct 15
Meet Scout 1.0, our first AI model built for investigating production incidents. Scout matches frontier-model investigation quality at a fraction of the cost, which makes it practical to investigate more issues, earlier. We trained Scout on realistic simulations built from real-world incidents and production telemetry, using reinforcement learning to reward accurate root causes backed by evidence. Learn more and sign up for the preview: bit.ly/3TJknGO
1
12
76
5,390
Post-trained with SkyRL!
Brian Armstrong
@brian_armstrongOct 11
We were able to fine tune an open weight model (Qwen3.5-9B) on proprietary internal data at @coinbase, and it out performed frontier models, at much lower cost and speed, for a specialized task (fraud prevention in this case). I think we'll see more of this pattern inside companies. The frontier models are amazing generalists. But increasingly you'll see specialist models out perform on more narrow tasks, either working on teams, or by themselves. We're making it easy to fine tune models internally for our employees. We even have a skill for it in our internal harness that makes it easy to spin these up and start training them: "/fine-tune-and-serve-llm I want to fine-tune Qwen for payments risk review. help me get started" I imagine we'll have some backend services that could be replaced entirely by this new architecture.
3
50
2,068
NovaSky retweeted
Richard Liaw@richliawOct 12
Post-trained with SkyRL @NovaSkyAI Served on Ray Serve LLM @raydistributed all on Anyscale Platform @anyscalecompute
Brian Armstrong
@brian_armstrongOct 11
We were able to fine tune an open weight model (Qwen3.5-9B) on proprietary internal data at @coinbase, and it out performed frontier models, at much lower cost and speed, for a specialized task (fraud prevention in this case). I think we'll see more of this pattern inside companies. The frontier models are amazing generalists. But increasingly you'll see specialist models out perform on more narrow tasks, either working on teams, or by themselves. We're making it easy to fine tune models internally for our employees. We even have a skill for it in our internal harness that makes it easy to spin these up and start training them: "/fine-tune-and-serve-llm I want to fine-tune Qwen for payments risk review. help me get started" I imagine we'll have some backend services that could be replaced entirely by this new architecture.
4
21
1,451
NovaSky retweeted
AfterQuery
@AfterQueryOct 8
Read more about our experiment using SkyRL here: afterquery.com/blog/improvin… To learn more about our tasks and reward design, reach out to us at [email protected]

AfterQuery - Improving Qwen3.8-27B-Medium on DeepSWE with GRPO

GRPO on 500 SWE agent training tasks created by AfterQuery raised Qwen3.8-27B’s DeepSWE pass@1 (avg 3) from 27.8% to 39.1%.

afterquery.com
2
7
3
28
4,061
Kimi K2.5-2.7 can be trained with RL on just 16 B300 GPUs (yes, just 2 nodes!) with full context Most post-training frameworks out there will tell you that you need 16 full nodes, and might not even give you full context Thanks to SkyRL maintainers for collab
Eric Tang@erictang000Oct 2
Excited to announce the release of SkyRL v0.4.0! Some highlights: - Support for GLM 5.3 Flash, GLM 5.3, and Kimi K2.7 - Top-P sampler replay and optimized R3 data transfer - Full FP8/MXFP8 RL - Sharded P2P weight transfer Blog: blog.skyrl.ai/blog/skyrl-v0-… Details in 🧵
8
17
1
355
23,866
NovaSky retweeted
Eric Tang@erictang000Oct 2
Excited to announce the release of SkyRL v0.4.0! Some highlights: - Support for GLM 5.3 Flash, GLM 5.3, and Kimi K2.7 - Top-P sampler replay and optimized R3 data transfer - Full FP8/MXFP8 RL - Sharded P2P weight transfer Blog: blog.skyrl.ai/blog/skyrl-v0-… Details in 🧵
2
32
3
190
31,982
SkyRL v0.4.0 enables large scale RL training with features like top-p sampler replay, R3, and FP8 RL on new models like GLM 5.3 Flash from @Zai_org!
Eric Tang@erictang000Oct 2
Excited to announce the release of SkyRL v0.4.0! Some highlights: - Support for GLM 5.3 Flash, GLM 5.3, and Kimi K2.7 - Top-P sampler replay and optimized R3 data transfer - Full FP8/MXFP8 RL - Sharded P2P weight transfer Blog: blog.skyrl.ai/blog/skyrl-v0-… Details in 🧵
2
16
1,424
NovaSky retweeted
Edward Hu
@edwardjhuSep 1
How does one RL post-train a 397B model for long-horizon knowledge work? 👩‍💼 We share every step we took to bring Qwen 3.5 397B from 16.1% Pass@1 to 27.3% on APEX-Agents using DPPO, including final models weights and the full training script🚀 This is the first of many works from Mercor Research on open model training research. Full blog: mercor.com/blog/training-fro… Source code: github.com/Mercor-Intelligen…
16
117
24
792
156,983
Load more