Rich Hastings

AI startup Cohere found that $AMZN Trainium 1 and Trainium 2 chips were underperforming $NVDA H100 GPUs According to an internal confidential Amazon document from July obtained by Business Insider. Cohere reported that access to Trainium 2 was extremely limited and affected by frequent service disruptions, the document noted. Stability AI, a startup known for generating AI images, had similar concerns. It concluded that Trainium 2 underperformed the H100 on latency, making it less competitive in both speed and cost. An $AMZN spokesperson said the company is grateful for customer feedback that helps make its chips even better and more widely used. The Cohere case is not current, the spokesperson added, noting that Trainium and Inferentia have achieved great results with customers including Ricoh, $DDOG, and Metagenomi. “We expect to accommodate more customers starting with Trainium 3, previewing later this year,” the spokesperson said. “We’ll accomplish that as we always do by listening to our customers and remaining vocally self-critical as we continue innovating to give more customers access to chips with the best possible price and performance.” Last week, $AMZN CEO Andy Jassy said during the company’s earnings call that Trainium 2 chips are fully subscribed and represent a multibillion-dollar business growing 150% quarter-over-quarter. Spokespeople for Cohere and Stability AI didn’t respond to requests for comment. Other AWS customers have also raised complaints about $AMZN AI chips. According to the July document, a startup called Typhoon found $NVDA A100 GPUs to be up to three times more cost-efficient than AWS Inferentia 2 chips for certain workloads. Similarly, a research group called AI Singapore determined that AWS G6 servers, equipped with $NVDA GPUs, offered better cost performance than Inferentia 2 across multiple use cases. Inferentia chips are used for inference, while Trainium chips focus on training models. A new $38 billion partnership between AWS and OpenAI highlights $AMZN ongoing challenges. The deal involves AI cloud servers that only contain $NVDA GPUs, with no mention of Trainium processors.