Speech + LLMs @nvidia | Previously: @Meta MSL, @jhuclsp, @IITGuwahati

New York, NY
Joined September 2009
NVIDIA's Nemotron 3.5 Transcribe ties for #1 among open weights Speech to Text models with 2.8% WER, using just 4B active parameters Nemotron 3.5 Transcribe is NVIDIA's new open weights Speech to Text model for non-streaming transcription, with 30B total and 4B active parameters, and the most accurate NVIDIA model on our non-streaming Speech to Text leaderboard. Key takeaways ➤ Accuracy: Nemotron 3.5 Transcribe scores 2.8% on the AA-WER Index, tied with Mistral's Voxtral Small for #1 among open-weight models. It leads Voxtral Small on AA-AgentTalk, our dataset of voice agent conversations, at 1.8% vs. 2.3%. ➤ Model size: With 4B active parameters (30B total), Nemotron 3.5 Transcribe sits on the open weights frontier of AA-WER vs. active parameters. It matches the accuracy of Mistral's Voxtral Small with 6x fewer active parameters (4B vs. 24B), while other open weights models of a similar size sit at around 4% WER. Congratulations to @NVIDIAAI on the release! See more details below ⬇️
43
26
11
446
67,985
Desh Raj retweeted
Meet NVIDIA Nemotron 3.5 Transcribe 🎙️ An open-weight, promptable 30B MoE model with ~4B active parameters that brings transcription in 40 languages, speaker attribution, translation, contextual prompting and audio understanding into one checkpoint. Watch it keep track of who said what when everyone’s talking at once. 1/ 🧵
188
393
201
5,188
455,437
Saturday is for time with family.
Saturday is for fixing bugs while sipping coffee. What are some great places to hangout in SF?
46
63
13
2,586
176,680
When we unveiled Moshi two years ago, we also showed an early checkpoint we'd trained just for fun, directly on Fisher (phone calls from the late 90s and early 2000s). As a result, every time you called, you'd get to chat with a random person straight out of that era. That was trained on just 2k hours of data. Today, you can get 100s of thousands of hours of full-duplex conversational data. There's an obvious scaling law at play here!
We believe voice is the most natural way to interact with intelligence, so we created Dessa, the most authentic full-duplex speech model ever built. Talking with it feels like talking with a person…full of emotion, nuance, pauses, and a natural cadence. block.xyz/dessa
15
19
205
12,918
Desh Raj retweeted
idk I try to help youngsters high schoolers etc in my DMs if I can be helpful this is the reason why I don't like SF, feels like it's full of social hierarchies where numbers/markers matter not as much in Europe
people need to status max so much more high status people saying "chasing status is bad" is such an effective way of keeping everyone below them literally the only reason why you care about a person you have not met is because they've done something publicly approved rant:
7
7
140
9,216
Desh Raj retweeted
Meet NVIDIA Nemotron 3.5 Super VL Our new open model combines reasoning and tool use with text, image, document, and video understanding for long-running AI agents. 120B total parameters with 12B active 🧵
🤖 Made with AI
290
553
365
4,457
603,600
How to kill a voice agent? Just turn on the TV ☠️ We benchmarked 14 voice agents with a TV playing in the background. Task completion fell from 38.7% to 8.6% 🧵/
29
19
13
192
33,100
Desh Raj retweeted
Introducing ChatGPT for Dishwashing GPT-6 achieves state-of-the-art results on dishwashing benchmarks for long-running jobs and difficult stains. The Navier–Stokes Millennium Prize Problem is a $1 million challenge about whether the equations describing fluid motion can break down. In this video, I accidentally found a counter-example while trying to optimize my dishwasher. But in reality, OpenAI used 10,000 AI agents to find a proof in 88 hours: openai.com/index/navier-stok… Parody of the original GPT-6 Astra ad @OpenAI #ChatGPT_Partner #chatgpt Credits: Written and Directed by Joma Co-Writer - Henry Connor Coan IV @coan_henry Production by Dauntlus Studios @dauntlus.studios Producer/Co-Director - Marcus Liew @dauntlus Camera Operator - Amos Lee @amoselijahlee Art Director - Alethea Soo @aletheasoo Art Assistant - Alicia Lim @aliciatkl_ Assistant Editor - GPT-6 Astra Music: Tensions Run High by Soundridemusic youtube.com/watch?v=Ly9H63SL…
423
960
341
10,790
1,128,548
love how in 90s and 2000s romcoms, nyc was never just a backdrop but a character of its own
104
5,444
297
46,113
1,354,937
Desh Raj retweeted
We’re looking for our next cohort of graduate fellows. Our 2027–2028 Graduate Fellowship Program is open to Ph.D. students worldwide with awards up to $60,000, plus mentorship and technical support. Apply by October 30: nvda.ws/4rGKDhv
🤖 Made with AI
10
108
8
595
116,804
Sigh. I wrote this post below, but then I didn't post it because I am tired of the consistent pushback I get from folks with the same safety worldview as Anthropic. I guess that means I should send it. Here goes! This post is pretty solipsistic and doesn't properly take recent events in cyber risks into it's discussion. It is really hard for me to watch how the US frontier labs like Anthropic don't have the ability to consider other approaches to safety and ways things could play out. It insinuates that Z ai (Chinese lab, builds GLM series) doesn't really care about safety and is reckless to take their business strategy. Saying things like "Given this evidence, we think it's likely both state and non-state actors will use models like GLM-5.3 to cause real-world harm." and "This is unlike any other similarly capable AI model, all of which were released with safeguards or through limited access programs." while closed models have been used on more of the documented cyber attacks is just bowing out of the interesting question. A plausible view is that open model weights and closed model apis (with some safe guards) are both far closer to being easy to mis-use, rather than API models being closer to safe. The trope "Open Dangerous, Closed Safe" may be closer to "Open Unsafe, Closed Unsafe" Closed models have stronger capabilities and stronger safeguards, but the stronger capabilities part could matter more in net harm if both the safeguards are porous. At the same time, as the authors do acknowledge (thanks - thats progress!), open models without extreme cyber guardrails are important to rapidly diffuse cyber readiness in the economy -- as programs like project glasswing are not perfect in getting all critical industry onboarded. There will be more issues like the time when Fable was released and Amazon found a workaround, which allowed them to access the full capabilities of the model served readily at an API. The blog overall is reasonable in it's narrow line, but it's a very effective tool in a complicated, rapidly evolving media ecosystem to reinforce a certain type of safety thinking.
52
86
16
696
77,874
Who wants more speech training data? YODAS v3 is now available @huggingface It’s 1.1M hours - the biggest audio dataset ever. And the first at this scale with stereo audio at 48kHz, with timestamped transcripts and translations. 100+ langs hf.co/blog/espnet/yodasv3 CC-BY-3.0
23
73
13
457
49,468
The phrase "stochastic parrot" is full of sound and fury, signifying nothing. Which, ironically, is exactly what the authors got wrong about language, and the core technical mistake of the paper. 1/
Everyone! This is a straw-person argument. The Stochastic Parrot paper was about LLMs of 2021, not the AI of today, which are not LLMs but complex software systems with vast post training and many external software components.
21
42
9
368
91,372
Sydney, Australia 🐨🇦🇺 #Interspeech2026 We’re presenting 7 papers this year! I’m also looking for motivated PhD students to join my lab at JHU CLSP in Fall 2027. If you’re interested, feel free to come find me at our paper presentations. (Check Interspeech app for details)
1
5
28
1,171
Nemotron-3-Diarization: our latest SOTA diarization model, as ranked on Voice Arena Diarization. 🗣️up to 8 speakers 🌎21 languages 🔥15000 RTFx ⚡️offline and online 🕕variable latency down to 0.32s 💰commercial and non-commercial use huggingface.co/blog/nvidia/n…
10
21
3
279
13,161
NVIDIA Nemotron 3 Diarization is now available on ModelScope—adding live speaker attribution to existing ASR workflows without replacing the transcription model. 🎙️ 🤖 modelscope.ai/models/nv-comm… 👥 Processes streaming audio and returns speaker labels and timestamps for up to eight speaker slots in a single conversation. ⚡ Its end-to-end streaming architecture avoids separately combining voice activity detection, speaker embeddings, clustering, and post-processing. 🧠 The 99.2M-parameter model uses a 31-layer Transformer encoder with RoPE and builds on NVIDIA’s Streaming Sortformer architecture. 🔌 Pair it with Nemotron ASR, Parakeet, Canary, Whisper, or another ASR system to create speaker-attributed transcripts. 🏢 Designed for meetings, contact centers, clinical conversations, live captioning, media analysis, and multi-party voice agents. 🖥️ Supports NVIDIA Ampere, Hopper, and Blackwell GPUs, with inference through NeMo Speech C++.
🤖 Made with AI
3
10
1
71
5,109
Desh Raj retweeted
When several people talk at once, a transcript can get messy fast. Our new Nemotron 3 Diarization model tracks who spoke when, even when voices overlap. It handles up to eight speakers, has 100M parameters, and is now available on @huggingface 🤗
205
856
240
9,691
1,068,160
A long-held dream, finally real: JSALT 2027 comes to Japan!! Tohoku University, Sendai, 14 Jun – 6 Aug 2027. One problem, one team drawn from academia, industry and government, in residence. A first in Japan in three decades. 1-page proposals due 16 Oct: forms.gle/Q6ZSxkJtLLRZcnJQ9
1
28
2
100
9,663
I'm super bored of LLM-stylized papers/blogs. I think we all are? So I asked Claude to explain its tells: web.mit.edu/phillipi/www/cla… Let's avoid these! I'd much rather read *your* words, see *your* rickety figures, than face down yet another of Claude's anodized corporate veneers.
26
40
6
602
49,697