inference providers have to choose to be either fast or cheap if you choose speed as a provider, you'll have idle gpus. but if you optimize to be cheap, customers are going to be waiting the pay-as-you-go pricing model means you pay per token, and the provider pays for gpu time. being fast means putting fewer people on each gpu, so each token costs more groq and cerebras chose to be fast. they keep the # of people they serve per gpu low coreweave, novita, and crusoe chose to be cheap. they charge much less this is just how gpus work. on a blackwell, if one person uses the gpu alone, they get 1,000 tokens a second, and the gpu produces 1,000 tokens a second in total because one person can't keep it busy if many people share the gpu, it produces 60,000 tokens a second in total, but each person only gets about 20 to 100 of those