Control plane for agents & engineers to provision compute and run training & inference across NVIDIA, AMD, and other chips — on clouds, Kubernetes, and on-prem.
github.com/dstackai/dstack
Joined January 2020
- Tweets1.3K
- Following55
- Followers1.6K
- Likes1.1K
dstack retweeted
Biggest learning at PyTorchCon this week is that engineers are so hungry for alternatives to Slurm and Kubernetes. As soon as you mention it, you get their attention.
Super excited about the conference! Also great to see at least 5 GPU hardware vendors here. It’s the first time I’ve seen so many at such a small event. Exciting times for inference ahead!
dstack retweeted
In San José tonight? Come hang out! We’re throwing a @PyTorch Conference afterparty with @CrusoeAI and the @lmsysorg SGLang
luma.com/6j69xyal
dstack retweeted
Looking forward to @PyTorch Conf this week. Hanging out at the @dstackai booth, and Tue evening at our meetup with @CrusoeAI and @lmsysorg SGLang: luma.com/6j69xyal
In San Jose and SF all week, happy to grab coffee.
dstack retweeted
dstack 0.22.4 is out!
This release greatly improves server stability and performance, with significantly faster offer lookups and better responsiveness under heavy workloads.
Also included: UI improvements to Launch, Offers, and project quotas in dstack Sky.
github.com/dstackai/dstack/r…
dstack retweeted
How many coding agents can one 4xH200 node serve with Qwen3.8-Flash-Next?
With dstack presets, we tuned @lmsysorg's SGLang on Claude Code sessions: 37% lower TTFT, 74% lower p99 ITL vs the cookbook recipe.
Read about the benchmark and how preset optimization works: dstack.ai/blog/qwen38-flash-…
Huge thanks to @jarvislabsai for partnering!
dstack retweeted
Open Source AI Week brings us to San Jose on Oct. 20 for AI Infra Night with @sgl_project and @dstackai.
Come meet GPU, ML, and infra engineers after the PyTorch Conference day, and request your spot early!
Where to find #SGLang at Open Source AI Week:
Sun Oct 18 · InclusionAI Tech Night, SF
Tue Oct 20 · SGLang x LinkedIn Meetup, Sunnyvale
Tue Oct 20 · AI Infra Night at PyTorchCon, San Jose
Links below. opensourceaiweek.com @open_intel_fdn #OpenSourceAIWeek #OpenIntelligence
1️⃣ Sun Oct 18 · InclusionAI Tech Night: Your AI Reality Check in SF, hosted by Ant Open Source and inclusionAI. Request to join: luma.com/xs9sn0lp
2️⃣ Tue Oct 20 · SGLang x LinkedIn Meetup in Sunnyvale. Talks on LLM serving, including how LinkedIn runs SGLang in production. Limited spots, request to join: luma.com/zbp6a1e1
3️⃣ Tue Oct 20 · AI Infra Night #PyTorchCon in San Jose with dstack and Crusoe. GPU and infra folks, after the conference day. Request to join: luma.com/6j69xyal
dstack retweeted
dstack 0.22.3 is out!
You can now provision @HotAisle bare metal servers with 8x @AMD MI300X GPUs, in addition to VMs. Bare metal support is experimental.
This release also includes a number of major bug fixes and improvements. Release notes: github.com/dstackai/dstack/r…
dstack retweeted
Sponsor news from @CrusoeAI: We’ll be in San Jose for #PyTorchCon North America Oct 20-21.
🎤 Talks on Triton, MoE fine-tuning + model customization
🤖 Booth demos + challenges
🍻 AI Infra Night with @dstackai, @sgl_project (@lmsysorg)
Learn more: bit.ly/3TKs7bi
Register for AI Infra Night: 🔗 bit.ly/4AX0nRy
#PyTorchFoundation
dstack retweeted
dstack 0.22.2 is out!
You can now provision compute on @daytonaio and orchestrate training and inference with NVIDIA B300/H200 and AMD MI355X GPUs.
Presets now use cache-aware routing even without PD disaggregation.
Release notes: github.com/dstackai/dstack/r…
dstack retweeted
We’ll be in San Jose for @PyTorch Conference North America Oct 20 - 21.
🎤 Talks on Triton, MoE fine-tuning + model customization
🤖 Booth demos + challenges
🍻 AI Infra Night with @dstackai, @SGLang (@lmsysorg)
Register: 🔗 luma.com/6j69xyal
#PyTorchCon #PyTorchFoundation
dstack retweeted
dstack 0.21.3 is out!
Presets: optimized inference configurations can now be pushed to a private registry and later deployed to any cloud, Kubernetes, or on-prem cluster.
Backends: @seeweblive GPU cloud is now supported.
Plus improvements to services and gateways. Release notes: github.com/dstackai/dstack/r…
dstack retweeted
Here's one example of using the presets toolkit to optimize Qwen3.8-27B on a single @AMD MI300X.
Through compound learning across linked sessions and source-level patches, performance improved from 311 to 495 tok/s: +59%.
The resulting preset can be deployed on any AMD cloud, Kubernetes cluster, or bare-metal fleet.
Inference serving is increasingly open source. But inference optimization still happens inside each provider, and the optimized deployment remains tied to its proprietary stack. This has to change.
Introducing Presets: an open-source toolkit for agent-based inference optimization and a portable format for the result.
Deploying optimized Kimi K3 to any cloud, Kubernetes cluster, or bare-metal fleet, whether on NVIDIA, AMD, or other silicon, should be as simple as deploying a Docker image.
dstack.ai/blog/presets/
dstack retweeted
Inference serving is increasingly open source. But inference optimization still happens inside each provider, and the optimized deployment remains tied to its proprietary stack. This has to change.
Introducing Presets: an open-source toolkit for agent-based inference optimization and a portable format for the result.
Deploying optimized Kimi K3 to any cloud, Kubernetes cluster, or bare-metal fleet, whether on NVIDIA, AMD, or other silicon, should be as simple as deploying a Docker image.
dstack.ai/blog/presets/
dstack retweeted
i've been saying for a while now that we need a good harness for tuning inference, but also deploying it.
@dstackai has picked up the ball and done it. glad to partner with them for years now. keep up the great work.
Inference serving is increasingly open source. But inference optimization still happens inside each provider, and the optimized deployment remains tied to its proprietary stack. This has to change.
Introducing Presets: an open-source toolkit for agent-based inference optimization and a portable format for the result.
Deploying optimized Kimi K3 to any cloud, Kubernetes cluster, or bare-metal fleet, whether on NVIDIA, AMD, or other silicon, should be as simple as deploying a Docker image.
dstack.ai/blog/presets/
dstack retweeted
dstack 0.21.1 is out!
Presets: the inference optimization agent can now patch source code (serving framework, kernels, etc). Also, sessions can be linked, so each new one starts from the previous best and pushes further.
Gateways: replicas are now fault tolerant and support in-place scaling.
Plus lots of other improvements: github.com/dstackai/dstack/r…
dstack retweeted
Love it! Works on @tenstorrent. Go @dstackai
Super excited about this release. Most of the work went into presets.
If you spend time optimizing inference, this is for you. You point the agent at a model and it figures out how to serve it best, down to patching kernels when flags aren't enough. A built preset can be deployed anywhere.
Works on @nvidia, @AMD, and @tenstorrent, any cloud, K8S, or bare metal.
dstack.ai/docs/concepts/pres…
A longer write-up with results from our own sessions is coming soon.
dstack retweeted
Super excited about this release. Most of the work went into presets.
If you spend time optimizing inference, this is for you. You point the agent at a model and it figures out how to serve it best, down to patching kernels when flags aren't enough. A built preset can be deployed anywhere.
Works on @nvidia, @AMD, and @tenstorrent, any cloud, K8S, or bare metal.
dstack.ai/docs/concepts/pres…
A longer write-up with results from our own sessions is coming soon.
dstack 0.21.1 is out!
Presets: the inference optimization agent can now patch source code (serving framework, kernels, etc). Also, sessions can be linked, so each new one starts from the previous best and pushes further.
Gateways: replicas are now fault tolerant and support in-place scaling.
Plus lots of other improvements: github.com/dstackai/dstack/r…
dstack retweeted
Autoresearch is a real shift in how AI itself is built. @transformerlab just launched Primus, a service that automates the full research loop.
It reads papers, forms hypotheses, writes code, runs experiments, and writes the final paper.
Also great to see @dstackai helping with compute orchestration!
I am deeply proud to finally share what we’ve been building. We're calling it Primus. With a single prompt, it does the full job of a Machine Learning researcher from prompt to paper.
Our goal is massive: we believe this can change the trajectory of how all scientific research is done in the future. That is a heavy statement, but we aren't making it lightly. Over the last few months, this system has entirely rewired how our own team works and thinks.
It is a profound shift to experience firsthand. We cannot wait for you to try it yourself --> visit our webpage to use it now.