Dev Patel (@devpatelio) header
Dev Patel (@devpatelio) profile picture
SF, CA
Joined June 2022
Photo by Dev Patel (@devpatelio)Photo by Dev Patel (@devpatelio)Photo by Dev Patel (@devpatelio)Photo by Dev Patel (@devpatelio)Photo by Dev Patel (@devpatelio)Photo by Dev Patel (@devpatelio)Photo by Dev Patel (@devpatelio)Photo by Dev Patel (@devpatelio)Photo by Dev Patel (@devpatelio)Photo by Dev Patel (@devpatelio)
Our work at @novaskyai with @afterquery on integrating SDPO into visual code generation is out! We explored how targeted feedback, teacher selection, and KL approximations affect sdpo training, with cool findings on stability and efficiency.
We added support for SDPO in SkyRL alongside UC Berkeley SkyLab researchers. Our new blog explores self-distillation across math, document, and chart-to-code tasks. On MATH-500, Qwen3-8B improved from 85.6% to 91.2% (+5.6 points), while using 8× fewer rollouts than GRPO. afterquery.com/blog/on-polic…
3
6
66
3,875
Dev Patel retweeted
1/8 Excited to announce the release of SkyRL v0.5.0! Some highlights: - Support for Beam-1.0, DeepSeek-V4.1-Flash, Qwen3.8-Flash-Next - Score centering for stable RL with 4-bit rollouts - 1M-token context training for GLM-5.3-Flash + optimizer offload to disk - A native On-Policy Distillation (OPD) trainer - Better VLM training: sequence packing, context parallelism and images in agentic RL Blog: blog.skyrl.ai/blog/skyrl-v0-… Details in 🧵
4
16
2
59
3,013
Dev Patel retweeted
Excited to announce Day-0 support for Inkling 2 Small from @thinkymachines on SkyRL! We’ve validated full MXFP8, BF16 and LoRA RL with @vllm_project and Megatron. Really excited about the customizability results and to see how the OSS community uses Inkling2!
3
15
3
41
2,622
Dev Patel retweeted
Super excited to share this work from our AWS FDE team! 🚀 With AgentCore RL Toolkit (ART) + SkyRL (@NovaSkyAI), a 9B SWE-Grep agent beats Claude Sonnet 5 on line F1 across two code-retrieval benchmarks after 30 RL steps. See it in action. 👇 1/6
4
9
3
40
6,083
im incredibly grateful for my time at thrive. the generosity, thoughtfulness, and rigor of the firm and its people is inspiring and unparalleled. being so close to greatness is incredibly rare and has taught me so much. apply! fellows.thrivecap.com/
at thrive, we deeply believe that the best ideas can come from anyone. our fellowship is one way we put it into practice. each year, we support a small group of college students as they pursue their own ideas alongside our team. we have been fortunate to learn from those who have shared their time and experience with us. we do not have all the answers, but we hope that sharing what we have learned can help our fellows find their own. fellows.thrivecap.com/
2
59
3,637
Lots of great topics. This covers - SkyRL at Mercor - What’s new in SkyRL - 1M-context training - Token-in token-out - Ray Sandboxes - Harbor - Terminal-Bench
7
111
10,837
Dev Patel retweeted
Super excited to share some of the work I’ve been doing on supporting 1M token context training on SkyRL for GLM 5.3 Flash, 64x higher than the out of the box Megatron baseline! 🚀 This work was done in collaboration with the amazing folks at @LilaSciences! 🤩 Getting there took key algorithmic and systems optimizations in four main areas: 1) DeepSeek Sparse Attention: run DSA on a sparse kernel; chunk and shard its quadratic indexer. 2) Activation recompute and offloading: recompute activations in backward and offload saved inputs to CPU. 3) Context parallelism: split each sequence across GPUs, with new CP support for KDA and a fix for DSA. 4) Avoiding unnecessary precision upcasts: keep FP32 copies out of memory and off the network. For more details on our implementations and our results, check out our full technical blog here: blog.skyrl.ai/blog/1m-contex…! This work benefited from the contributions of many people along the way. Big thanks to @pcmoritz, @erictang000, @sumanthrh, @CyrusHakha, @richliaw from @anyscalecompute and Keith Truongcao, Tushar Khot, Vivek Subramanian, Semih Kara from Lila Sciences for all the help and guidance!
5
26
6
174
17,282
Choosing the right AI model is an operating decision. Today, we’re sharing our first set of benchmarks drawn from work across the 75+ accounting and IT services firms that make up Thrive Holdings. We believe the field needs more benchmarks tied to economically valuable work, so builders can understand where models are useful in real-world environments. We use these tests to help us compare model configurations and choose which models our systems run on, with the goal of delivering better service to our customers. We’ll add to the benchmarks as we build new products and expand to additional industries. evals.thriveholdings.com/
3
19
11
141
26,848
Dev Patel retweeted
Meet Scout 1.0, our first AI model built for investigating production incidents. Scout matches frontier-model investigation quality at a fraction of the cost, which makes it practical to investigate more issues, earlier. We trained Scout on realistic simulations built from real-world incidents and production telemetry, using reinforcement learning to reward accurate root causes backed by evidence. Learn more and sign up for the preview: bit.ly/3TJknGO
25
66
36
335
52,922
Announcing the beta release of the Pi API. We’ve partnered with companies across diverse industries to adapt our models to their robots and applications, from harvesting strawberries to assembling servers. Now we’re opening access to more teams.
142
250
230
2,380
436,231
We were able to fine tune an open weight model (Qwen3.5-9B) on proprietary internal data at @coinbase, and it out performed frontier models, at much lower cost and speed, for a specialized task (fraud prevention in this case). I think we'll see more of this pattern inside companies. The frontier models are amazing generalists. But increasingly you'll see specialist models out perform on more narrow tasks, either working on teams, or by themselves. We're making it easy to fine tune models internally for our employees. We even have a skill for it in our internal harness that makes it easy to spin these up and start training them: "/fine-tune-and-serve-llm I want to fine-tune Qwen for payments risk review. help me get started" I imagine we'll have some backend services that could be replaced entirely by this new architecture.
614
787
547
9,457
1,613,740
Dev Patel retweeted
Not all coding data is created equal. Using only 500 tasks from @AfterQuery's SWE agent dataset, our internal post-training team was able to improve Qwen3.8-27B-Medium by 11.3 points in only 15 steps of GRPO. Read more about our training experiment in the blog post linked in the comments.
27
32
7
216
32,131
Replying to @snbafana
🌱
1
4,022
Dev Patel retweeted
And that’s the story! Along the way, we’ve upstreamed several OSS contributions that you can find in SkyRL. If you’re interested in learning more, read the full write up here: trajectory.ai/field-notes/en… Here's to many more expeditions 🧭
3
15
1,180
Dev Patel retweeted
SkyRL v0.4.0 enables large scale RL training with features like top-p sampler replay, R3, and FP8 RL on new models like GLM 5.3 Flash from @Zai_org!
Excited to announce the release of SkyRL v0.4.0! Some highlights: - Support for GLM 5.3 Flash, GLM 5.3, and Kimi K2.7 - Top-P sampler replay and optimized R3 data transfer - Full FP8/MXFP8 RL - Sharded P2P weight transfer Blog: blog.skyrl.ai/blog/skyrl-v0-… Details in 🧵
2
16
1,548
Dev Patel retweeted
Excited to announce the release of SkyRL v0.4.0! Some highlights: - Support for GLM 5.3 Flash, GLM 5.3, and Kimi K2.7 - Top-P sampler replay and optimized R3 data transfer - Full FP8/MXFP8 RL - Sharded P2P weight transfer Blog: blog.skyrl.ai/blog/skyrl-v0-… Details in 🧵
2
32
3
191
32,506