
Speech + LLMs @nvidia | Previously: @Meta MSL, @jhuclsp, @IITGuwahati
New York, NY
Joined September 2009
- Tweets2.4K
- Following2.3K
- Followers4.6K
- Likes8.6K
Desh Raj retweeted
NVIDIA's Nemotron 3.5 Transcribe ties for #1 among open weights Speech to Text models with 2.8% WER, using just 4B active parameters
Nemotron 3.5 Transcribe is NVIDIA's new open weights Speech to Text model for non-streaming transcription, with 30B total and 4B active parameters, and the most accurate NVIDIA model on our non-streaming Speech to Text leaderboard.
Key takeaways
➤ Accuracy: Nemotron 3.5 Transcribe scores 2.8% on the AA-WER Index, tied with Mistral's Voxtral Small for #1 among open-weight models. It leads Voxtral Small on AA-AgentTalk, our dataset of voice agent conversations, at 1.8% vs. 2.3%.
➤ Model size: With 4B active parameters (30B total), Nemotron 3.5 Transcribe sits on the open weights frontier of AA-WER vs. active parameters. It matches the accuracy of Mistral's Voxtral Small with 6x fewer active parameters (4B vs. 24B), while other open weights models of a similar size sit at around 4% WER.
Congratulations to @NVIDIAAI on the release!
See more details below ⬇️
Meet NVIDIA Nemotron 3.5 Transcribe 🎙️
An open-weight, promptable 30B MoE model with ~4B active parameters that brings transcription in 40 languages, speaker attribution, translation, contextual prompting and audio understanding into one checkpoint.
Watch it keep track of who said what when everyone’s talking at once.
1/ 🧵
Desh Raj retweeted
When we unveiled Moshi two years ago, we also showed an early checkpoint we'd trained just for fun, directly on Fisher (phone calls from the late 90s and early 2000s). As a result, every time you called, you'd get to chat with a random person straight out of that era. That was trained on just 2k hours of data. Today, you can get 100s of thousands of hours of full-duplex conversational data. There's an obvious scaling law at play here!
We believe voice is the most natural way to interact with intelligence, so we created Dessa, the most authentic full-duplex speech model ever built. Talking with it feels like talking with a person…full of emotion, nuance, pauses, and a natural cadence.
block.xyz/dessa
Desh Raj retweeted
idk I try to help youngsters high schoolers etc in my DMs if I can be helpful
this is the reason why I don't like SF, feels like it's full of social hierarchies where numbers/markers matter
not as much in Europe
Desh Raj retweeted
How to kill a voice agent? Just turn on the TV ☠️
We benchmarked 14 voice agents with a TV playing in the background.
Task completion fell from 38.7% to 8.6%
🧵/
Introducing ChatGPT for Dishwashing
GPT-6 achieves state-of-the-art results on dishwashing benchmarks for long-running jobs and difficult stains.
The Navier–Stokes Millennium Prize Problem is a $1 million challenge about whether the equations describing fluid motion can break down. In this video, I accidentally found a counter-example while trying to optimize my dishwasher. But in reality, OpenAI used 10,000 AI agents to find a proof in 88 hours: openai.com/index/navier-stok…
Parody of the original GPT-6 Astra ad
@OpenAI #ChatGPT_Partner #chatgpt
Credits:
Written and Directed by Joma
Co-Writer - Henry Connor Coan IV @coan_henry
Production by Dauntlus Studios @dauntlus.studios
Producer/Co-Director - Marcus Liew @dauntlus
Camera Operator - Amos Lee @amoselijahlee
Art Director - Alethea Soo @aletheasoo
Art Assistant - Alicia Lim @aliciatkl_
Assistant Editor - GPT-6 Astra
Music: Tensions Run High by Soundridemusic
youtube.com/watch?v=Ly9H63SL…
I think I'm going to get a lot of mileage out of this one.
🤖 Made with AI
Mathematicians are coping so hard they might abolish irrational numbers again and turn the field into a guild
terrytao.wordpress.com/2026/…
Desh Raj retweeted
love how in 90s and 2000s romcoms, nyc was never just a backdrop but a character of its own
We’re looking for our next cohort of graduate fellows.
Our 2027–2028 Graduate Fellowship Program is open to Ph.D. students worldwide with awards up to $60,000, plus mentorship and technical support.
Apply by October 30: nvda.ws/4rGKDhv
🤖 Made with AI
Desh Raj retweeted
Sigh.
I wrote this post below, but then I didn't post it because I am tired of the consistent pushback I get from folks with the same safety worldview as Anthropic. I guess that means I should send it. Here goes!
This post is pretty solipsistic and doesn't properly take recent events in cyber risks into it's discussion. It is really hard for me to watch how the US frontier labs like Anthropic don't have the ability to consider other approaches to safety and ways things could play out.
It insinuates that Z ai (Chinese lab, builds GLM series) doesn't really care about safety and is reckless to take their business strategy.
Saying things like "Given this evidence, we think it's likely both state and non-state actors will use models like GLM-5.3 to cause real-world harm." and "This is unlike any other similarly capable AI model, all of which were released with safeguards or through limited access programs." while closed models have been used on more of the documented cyber attacks is just bowing out of the interesting question.
A plausible view is that open model weights and closed model apis (with some safe guards) are both far closer to being easy to mis-use, rather than API models being closer to safe. The trope "Open Dangerous, Closed Safe" may be closer to "Open Unsafe, Closed Unsafe"
Closed models have stronger capabilities and stronger safeguards, but the stronger capabilities part could matter more in net harm if both the safeguards are porous.
At the same time, as the authors do acknowledge (thanks - thats progress!), open models without extreme cyber guardrails are important to rapidly diffuse cyber readiness in the economy -- as programs like project glasswing are not perfect in getting all critical industry onboarded.
There will be more issues like the time when Fable was released and Amazon found a workaround, which allowed them to access the full capabilities of the model served readily at an API.
The blog overall is reasonable in it's narrow line, but it's a very effective tool in a complicated, rapidly evolving media ecosystem to reinforce a certain type of safety thinking.
Desh Raj retweeted
Who wants more speech training data?
YODAS v3 is now available @huggingface
It’s 1.1M hours - the biggest audio dataset ever. And the first at this scale with stereo audio at 48kHz, with timestamped transcripts and translations. 100+ langs
hf.co/blog/espnet/yodasv3
CC-BY-3.0
Desh Raj retweeted
The phrase "stochastic parrot" is full of sound and fury, signifying nothing. Which, ironically, is exactly what the authors got wrong about language, and the core technical mistake of the paper. 1/
Desh Raj retweeted
Sydney, Australia 🐨🇦🇺 #Interspeech2026
We’re presenting 7 papers this year! I’m also looking for motivated PhD students to join my lab at JHU CLSP in Fall 2027. If you’re interested, feel free to come find me at our paper presentations.
(Check Interspeech app for details)
Desh Raj retweeted
Nemotron-3-Diarization: our latest SOTA diarization model, as ranked on Voice Arena Diarization.
🗣️up to 8 speakers
🌎21 languages
🔥15000 RTFx
⚡️offline and online
🕕variable latency down to 0.32s
💰commercial and non-commercial use
huggingface.co/blog/nvidia/n…
Desh Raj retweeted
NVIDIA Nemotron 3 Diarization is now available on ModelScope—adding live speaker attribution to existing ASR workflows without replacing the transcription model. 🎙️
🤖 modelscope.ai/models/nv-comm…
👥 Processes streaming audio and returns speaker labels and timestamps for up to eight speaker slots in a single conversation.
⚡ Its end-to-end streaming architecture avoids separately combining voice activity detection, speaker embeddings, clustering, and post-processing.
🧠 The 99.2M-parameter model uses a 31-layer Transformer encoder with RoPE and builds on NVIDIA’s Streaming Sortformer architecture.
🔌 Pair it with Nemotron ASR, Parakeet, Canary, Whisper, or another ASR system to create speaker-attributed transcripts.
🏢 Designed for meetings, contact centers, clinical conversations, live captioning, media analysis, and multi-party voice agents.
🖥️ Supports NVIDIA Ampere, Hopper, and Blackwell GPUs, with inference through NeMo Speech C++.
🤖 Made with AI
When several people talk at once, a transcript can get messy fast.
Our new Nemotron 3 Diarization model tracks who spoke when, even when voices overlap. It handles up to eight speakers, has 100M parameters, and is now available on @huggingface 🤗
Desh Raj retweeted
A long-held dream, finally real: JSALT 2027 comes to Japan!!
Tohoku University, Sendai, 14 Jun – 6 Aug 2027. One problem, one team drawn from academia, industry and government, in residence. A first in Japan in three decades.
1-page proposals due 16 Oct: forms.gle/Q6ZSxkJtLLRZcnJQ9
Desh Raj retweeted
I'm super bored of LLM-stylized papers/blogs. I think we all are?
So I asked Claude to explain its tells: web.mit.edu/phillipi/www/cla…
Let's avoid these! I'd much rather read *your* words, see *your* rickety figures, than face down yet another of Claude's anodized corporate veneers.

































