An inference optimization lab. We make models run faster and cheaper through compiler and serving research.

Mountain View, CA
Joined March 2024
Our hackathon is live: build the part of a GPU compiler that writes PTX, the assembly language of @nvidia GPUs, by Tuesday. The 15 problems are public, and we run your code on our GPUs, so a laptop is enough. $2,750 in prizes, and standout entries can lead to an interview.
1
1
1
163
Our hackathon starts this Wednesday: seven days to build the part of a GPU compiler that writes PTX, the assembly language of @nvidia GPUs. You write Python and we run your code on our GPUs, so you don't need one. $2,750 in prizes, and standout entries can lead to an interview.
1
1
91
More RTX 5090s are now available on the CloudRift marketplace, hosted in South Korea. Each @NVIDIAGeForce RTX 5090 has 32GB of VRAM, enough to serve a 32B model at 4-bit with room left for context. They're on demand and billed per second.
2
1
13
678,359
$0.65/hr on demand, with 10% off on a one-month reservation and 15% off on three months. Launch one from the console: console.cloudrift.ai
1
731
Back in 2018 our founder @ditrifonov and a friend started building the lowering stack for the Apple Neural Engine, with almost no compiler experience. In this lecture he builds a GPU compiler from scratch, from one PyTorch op all the way down to CUDA.
If you're curious to learn about GPU compiler fundamentals, check out my latest stream at GPU_MODE: Building a GPU Compiler from Scratch; Lowering PyTorch Graphs to CUDA via Multi-Level IRs. Thanks, @marksaroufim, for the opportunity! Join @CloudRiftAI upcoming hackathon to dive deeper! youtu.be/KK6TWf4Pp6A?si=KRxn…
1
1
140
Research we helped fund is out: Soroush Vahidi at @NJIT used our AI Builder Grant to study two questions: how to make good decisions when LLM inference runs on tight budget, and whether rankings of LLM serving schedulers hold up across workloads, regions and metrics. Both papers are now public.
1
66
Your coding agent can now find your GPU rentals and SSH into them. The new CloudRift CLI ships a skill for Claude Code, Codex, Cursor, Copilot, Gemini CLI and OpenCode. Run rift agent install once, then ask your agent to check the hardware or run a job on your box.
1
54
Build the part of a GPU compiler that generates #PTX, in one week, without owning a GPU. The emmy #hackathon runs October 21 to 27, remote and open worldwide. $2,750 in prizes, and standout entries can lead to an interview with our engineering team.
1
2
67
The @AMD MI350X is back on the CloudRift platform. At 288GB of HBM3e per GPU, a 70B model in FP16 fits on a single card with room left for KV cache. Capacity is limited, and it's on demand with per-second billing.
1
1
27,148
RiftStack AI retweeted
Holy Sh*t. This is insane. I have no idea what Fable 5.5 will be like, but I think I was wrong about some assumptions regarding anthropic.
Claude Opus 5.5 takes the top spot on the Artificial Analysis Intelligence Index, along with a 20% price cut and larger cache hit discount Claude Opus 5.5 brings Anthropic to parity with GPT-6 Astra on evaluations like Terminal-Bench 4.0 and AutomationBench-AA, while extending Anthropic’s lead in agentic knowledge work. At max effort it scores 58 on the Artificial Analysis Intelligence Index, the highest score we have measured by several points. Anthropic has cut Opus pricing to $4/$20 per 1M input/output tokens (Opus 5: $5/$25) and cache reads from $0.50 to $0.20. Key takeaways: ➤ Consistent strong performance, with leading scores on six of the ten Intelligence Index evaluations: Humanity's Last Exam 61.4% (previous best 59.1%, Claude Fable 5.1), SciCode 66.9% (63.1%, Fable 5.1), GDPval-AA v2.1, AA-Briefcase v1.1, AA-Omniscience and AutomationBench-AA. On Terminal-Bench 4.0 it scores 59.6%, level with the leader GPT-6 Astra (xhigh) and +11 points over Opus 5. It remains slightly behind on CritPt, AA-LCR, and GDP.pdf ➤ Leads in agentic knowledge work: On AA-Briefcase, our private frontier knowledge work evaluation, it reaches an Elo of 1822. This is +143 over Fable 5.1, ahead on both analytical quality and presentation, and is the first time Anthropic has reached presentation quality surpassing GPT-5.6 Sol. This evaluation tests whether models can produce accurate and well-presented professional outputs using our open source reference agent harness, Stirrup ➤ Level with Opus 5 on cost per task despite 1.6x the output tokens: Opus 5.5 (max) uses ~119k output tokens per Intelligence Index task, against ~73k for Opus 5 (max), ~78k for Fable 5.1 (max) and ~27k for GPT-6 Astra (max) ➤ Four of five effort levels sit on the Intelligence vs Cost per Task frontier: Opus 5.5 max, xhigh, high, and medium all sit on the Pareto frontier, costing less or outperforming other models scoring 50+ (GPT-6 Astra, Claude Fable 5.1, and Claude Opus 5) Other model details: ➤ Context window: 1 million token context with image and text input support, unchanged from Opus 5 ➤ Pricing: $4/$20 per 1M input/output tokens, down 20% from $5/$25 for Opus 5. Cache writes $5 per 1M tokens for the 5 minute TTL, down from $6.25. Cache reads have been further discounted to $0.20 per 1M tokens, down 60% from Opus 5’s $0.50. This is a 95% discount compared to uncached input pricing, up from 90% on previous Opus models ➤ Effort settings: Five effort settings (low, medium, high, xhigh, and max). Intelligence Index evaluations were run at all five with Anthropic's default fallback enabled
80
76
9
1,714
123,967
RiftStack AI retweeted
I think it's time to hang up my Three.js hat. First of all, this is not a pity post—I'll be fine. I'm also not 100% confident about this. But I'm an over-sharer and like to talk out loud, so I thought I would share some thoughts on where I see things going and why I'm considering quitting Three.js development. Some signals I've noticed lately that have gotten me thinking: - Since Astra was released, I've probably seen at least 10 posts from different people that have achieved similar quality to Three.js Water Pro or better. - Someone downloaded a video recording of one my posts, fed it to Astra and recreated it with 90% accuracy - I've had less and less clients referrals - A Three.js developer I respect very much is retiring from the industry due to impact of AI and lack of interest - I've heard 2nd hand accounts of other developers losing majority of course/plugin sales due to AI - I've noticed Three.js posts have shifted from work created by seasoned developers to vibe-coded games and demos. Fundamentally, I no longer really see value in what I do when it can be replicated so easily. I've always enjoyed working on hard problems. Naturally, outsourcing the hard part to an agent takes away a lot of the enjoyment for me. High-level orchestration of agents does not tickle my brain in any special way. At the end of most days I feel unfulfilled and bored. I honestly feel like I've lost much of my critical thinking ability as I've stopped coding. I find it harder to focus on tasks and my attention-span and comprehension is noticeably dropping. My plan right now is to finish up Water Pro v4 and see how that performs. After that, I think it's time to start thinking about what else is out there for me. I'd love to hear thoughts from other developers & freelancers on the matter, positive or negative. Feel free to vent!
322
151
78
2,845
443,091