rcarrata (@rcarrata_) profile picture

Principal AI Architect - AI Business Unit at @RedHat

Joined July 2017
Photo by rcarrata (@rcarrata_)Photo by rcarrata (@rcarrata_)Photo by rcarrata (@rcarrata_)Photo by rcarrata (@rcarrata_)
The vLLM meetup is coming to Madrid! I’ll be giving one of the talks. Come chat about inference, multimodal models, agentic AI, llm-d and Kubernetes. There’ll be snacks and drinks too! November 12 · 18:00–22:00 MOA Institute luma.com/gl1d2loa
44
rcarrata retweeted
Hola, Madrid. vLLM and llm-d are coming to town: Red Hat and @nvidia are hosting an evening on open source inference, with @vllm_project, LLM Compressor, vLLM-omni, safe agents, and @_llm_d_ on Kubernetes. 12 November | MOA Institute | Register: luma.com/gl1d2loa
1
3
16
1,067
rcarrata retweeted
Great to see @MistralAI pushing real-time speech models forward with Voxtral Mini 4B Realtime. Day 0 support is already available in @vllm_project, and Day 1 experimentation is already possible with Red Hat AI Inference Server, enabling teams to quickly test streaming ASR workloads through the new realtime audio API with sub 500ms latency and multi-language support. At Red Hat AI, our focus is helping teams move quickly from model release to real experimentation, giving developers and platform teams an early path to test real-time transcription, voice assistants, and live captioning workloads on open infrastructure. This is the open ecosystem moving together: ✅ Mistral releasing production-ready open models ✅ vLLM enabling immediate serving support ✅ Red Hat AI helping teams experiment on Day 1 If you are exploring real-time audio AI, this is a great place to start: 🤗 Model: huggingface.co/mistralai/Vox… 📰 Announcement: mistral.ai/news/voxtral-tran… 🛠️ Run with Red Hat AI Inference Server: gist.github.com/dougbtv/f27d…
2
11
5
35
5,238
Just dropped FP8-quantized version of Qwen/Qwen3-VL-235B-A22B-Instruct. It cuts disk size and GPU memory needs by about 50%, while maintaining >99.6% accuracy. huggingface.co/RedHatAI/Qwen…
2
6
2
31
7,124
DeepSeek-V3.2-Exp is live in vLLM on day-0, with support for Hopper and Blackwell. It's now ready for experimentation with Red Hat AI. Key takeaways from today's blog:
2
1
10
616
🚀 Big news! The September '25 batch of Red Hat AI Validated Models is here! Our goal is to give you the confidence, predictability, and flexibility you need to deploy third-party generative AI models across the Red Hat AI platform. Why do you need validated models? Selecting the right LLM for your use case is hard. Enterprises often struggle with: 🤯 Model selection complexity ⚖️ AI resource planning 📉 Ensuring performance can be reliably reproduced You can't "guess" your way into production. That's where Red Hat AI's model validation process comes in. We simplify the selection by performing extensive testing for you: 📊 Performance Benchmarking (w/ GuideLLM) ✅ Accuracy Evaluations (w/ LM Evaluation Harness) 🔄 Reproducible Deployments (on vLLM) This provides clear capacity planning guidance for customers before they go to production. Now, meet the September '25 lineup, a diverse batch optimized for your enterprise workloads: · Kimi K2 Quantized INT4: A RAG powerhouse. Its massive context window is built for analyzing lengthy documents, like legal contracts or research papers. See it on Hugging Face: huggingface.co/RedHatAI/Kimi… · Gemma-3n 4B FP8 Dynamic: Google's next-gen efficient model. Perfect for summarization and on-device tasks. See it on Hugging Face: huggingface.co/RedHatAI/gemm… · DeepSeek-R1 INT4: An elite coding model for generating, completing, and debugging complex code. See it on Hugging Face: huggingface.co/RedHatAI/Deep… · Qwen 3 8B FP8 Dynamic: A versatile and powerful multilingual model from Alibaba Cloud, ideal for global chatbots and content creation. See it on Hugging Face: huggingface.co/RedHatAI/Qwen… Get started today. You can run these models now on Red Hat Inference Server v3.2.1 and Red Hat OpenShift AI v2.24. In addition to Hugging Face, you can also access them as container images from the Red Hat Container Registry. They'll also be in the RHOAI Model catalog (coming in November😉)! Stay tuned for the next batch of validated models coming next month, including models like gpt-oss and Qwen3-Coder.
2
5
14
8,314
Developer Day Milan happening now! Here's Roberto Carratalà fand Marco Caimi showing Llama Stack @vllm_project @_llm_d_ and MCP servers in action @RedHat_AI @openshift @rhdevelopers
1
7
17
786
BIG thanks to the folks at @T3chFest for a wonderful and community-first conference this week, we had a blast sharing some open source tools and AI models 🤖 for developers to save on costs and keep your data private! I’ll link the demo repository and slides below 👇
4
1
6
343
rcarrata retweeted
Seguimos con más charlas, ahora @borjaperfra en Track 1, @vicajilau en Track 2, @cedricclyburn y @rcarrata_ en Track 3 y @cesartuit en el Track 4. #t3chfest10
1
2
8
594
Check this out!! So cool! Awesome work Neural Magic team 👏👏
.@IBM's Granite models are known for efficiency and performance. Neural Magic team at @RedHat optimized them even further in Granite 3.1! 🪶 3.3x smaller ⚡ Up to 2.8x faster 🎯 99% accuracy retention Here’s how these improvements unlock efficient open-source AI deployments 🧵
125
.@IBM's Granite models are known for efficiency and performance. Neural Magic team at @RedHat optimized them even further in Granite 3.1! 🪶 3.3x smaller ⚡ Up to 2.8x faster 🎯 99% accuracy retention Here’s how these improvements unlock efficient open-source AI deployments 🧵
2
12
2
25
1,582
Is your service ready to become open? Join @radek_vokal @rcarrata_ @stefthewalter @simonsteinbeiss @TomasTomec for #DevConf_CZ 2023 opening session, Friday at 9:30am to deep-dive the world of #OpenSource Services. Register to attend: devconfcz2023.evenbrite.com
3
1
10
1,370
rcarrata retweeted
We're excited to welcome Yamandú Tellechea from Mutua Madrileña to the #RHSummit stage! Tune in live on May 23 at 9 AM EDT to hear about @la_Mutua's Event-Driven #Automation strategy: red.ht/3OuZsls @Ansible
4
15
6,134
GitOps Deployments in Multi-cluster Production Environments by @rcarrata_ , Red Hat at #cdCon #GitOpsCon sched.co/1LAFy @CDeliveryFdn @linuxfoundation @CloudNativeFdn
1
7
751
📌 #cdCon + #GitOpsCon Talk 📌 Managing GitOps Deployments in Multi-cluster Production Environments by @rcarrata_ from @Red Hat Tuesday, May 9 at 3:30pm PDT Details here: sched.co/1LAFy
2
3
567
Well, @rcarrata_ and I are back on the road!, we will be pleased to talk about GitOps Patterns for managing cloud-native applications. What a pleasure to share stage with him next Feb 24 at @cloudnativeams ! Tickets: buff.ly/3FmsLSr
Replying to @rcarrata_
@rcarrata_ from @RedHat and @kubenothing from @SUSE will also come to KCD Amsterdam! Their talk "GitOps Patterns for managing cloud-native applications" will show us lots of valuable tips-and-tricks Let's give them a warm welcome! Tickets: buff.ly/3FmsLSr
1
7
512
Replying to @rcarrata_
@rcarrata_ from @RedHat and @kubenothing from @SUSE will also come to KCD Amsterdam! Their talk "GitOps Patterns for managing cloud-native applications" will show us lots of valuable tips-and-tricks Let's give them a warm welcome! Tickets: buff.ly/3FmsLSr
3
1
6
980
rcarrata retweeted
Tekton is really proud to have become a Graduated Project at the @CDeliveryFdn!! 🐱🤖🎓 Thanks to all contributors who helped reach this milestone and thank you to the @CDeliveryFdn and TOC for the support! Blog post: tekton.dev/blog/2022/10/26/t…
5
38
5
91
Great talk about Securing GitOps Supply Chain with Sigstore and Kyverno - Roberto Carratala & Faz Sadeghi, Red Hat at ArgoCon 2022 sched.co/14lxH @rcarrata_ #argocon
4
6