
AI Agent & LLM inference enthusiast | ex Computer Science @USCViterbi|Software Engineer @awscloud
Los Angeles, CA ⇌ Seattle, WA
Joined June 2022
- Tweets2.4K
- Following868
- Followers8K
- Likes10.5K
Jeffro retweeted
Excited to see what the community has built with SGLang-Omni!
It’s a pleasure to introduce and demo Voxt, an open-source voice input tool for Mac.
Voxt has been actively developed for over six months, shaped by real-world use and feedback from more than a thousand users. With this update, we’re bringing SGLang Omni into Voxt’s inference stack to further improve local inference performance. We’ll also continue refining the overall user experience.
There are already plenty of voice input tools available. Typeless, Doubao Input Method, Zhipu Input Method, and WeChat Input Method all offer these capabilities. Voxt’s strength is its open, controllable software stack: we can continuously optimize everything from the application to inference, improving both performance and stability. With a fully local configuration, speech recognition and text refinement run entirely on your own computer, so your audio and transcriptions don’t need to be uploaded to a remote inference service.
Today, we’ll focus on how Voxt handles technical terminology, particularly model names, framework names, and English acronyms used in AI engineering. Basic ASR alone can easily misrecognize these terms.
Voxt combines hotword recognition, LLM-based text refinement guided by a personal system prompt, and inference performance optimizations. Together, these make voice input efficient, improve its handling of technical vocabulary, and adapt it to your own style of expression.
This demo runs on my M5 Pro Mac.
Speech recognition uses SGLang Omni’s MLX branch, while text refinement uses Qwen3.8 27B running locally through Ollama Serve. Both steps run on this machine.
Voxt also provides interfaces for both ASR and LLMs. You can choose remote speech recognition services and remote language models to suit your needs—you’re not limited to the local setup shown in this demo.
Xuesong and I have been using Voxt for over six months, and we’ve gone from everyday users to contributors. Integrating SGLang Omni marks another milestone in our collaboration.
Thank you to Guan Wei and everyone involved. We invite you to download Voxt, give it a try, and help us make it even better.
Acknowledgments: Guan Wei · Ye Xuesong · Liu Chonghan · Zhou Yuhan · Deng Jiaxin
Project: github.com/sgl-project/sglan…
Download: github.com/sgl-project/sglan…
Jeffro retweeted
We brought SGLang to NVIDIA Vera Rubin and accelerated Kimi K3 inference.
Working closely with @NVIDIA, we optimized attention, MoE, and speculative verification kernels on early-access Rubin hardware.
Highlights:
• Up to 20% faster FP8 MLA at batch 1 / 128K context
• 20% faster KDA verification, with bitwise-identical output
• 5.9% end-to-end inference speedup from MoE tail fusion, removing 276 kernel launches per decode step
SGLang also powers rollouts for Miles' end-to-end RL training on Rubin, including agentic RL with 64 concurrent sandboxes on the Vera CPU.
Full results and engineering details 👉 lmsys.org/blog/2026-10-09-ve…
Jeffro retweeted
This first SGLang Summit will gather people from a broader ecosystem. Join us to meet people, products, users from different AI verticals, frontier research, cool hardware invents for the new era, and hear updates and customer stories around! Huge thanks to the whole community for sponsorships ❤️
Announcing the first SGLang Summit: November 12–13 at Fort Mason, San Francisco.
Join us for two days of talks, discussions, and deep dives into frontier AI infrastructure, silicon, models, and applications.
Register: sglang.io/summit
Jeffro retweeted
Announcing the first SGLang Summit: November 12–13 at Fort Mason, San Francisco.
Join us for two days of talks, discussions, and deep dives into frontier AI infrastructure, silicon, models, and applications.
Register: sglang.io/summit
快来 SGLang Omni 与 StepAudio3 的活动!
非常感谢阶跃朋友们的邀请,许久前就听说车企有许多与阶跃合作,构造车端一体化交互模型。这次 StepAudio3 发布,期待阶跃的多模态矩阵更上一层楼!
这一次阶跃将发布 StepAudio3 全新模型,并且在旧金山展开一次线下的产品 + 开发者交流会,我们团队的顾景文 Jingwen Gu 会作为嘉宾分享我们 SGLang Omni 在全双工模型的通讯协议和模型设计,也会和阶跃的朋友们共同分享下一代多模态交互式模型的技术愿景,欢迎来玩!!!
Jeffro retweeted
We bumped a dependency and quietly got dumber. Same weights, same inputs for models, the whole diff was the order four numbers get added.
This serves as a reminder that we need to pay attention to every action we might consider harmless in codebase.
Jeffro retweeted
Join us for a technical workshop with @Aptos , where you will learn to create a smart contract in Move and develop an Onchain AI agent. The event will conclude with a mixer featuring networking opportunities, free swag, and food! Registration link below! #Web3 #AI #LosAngeles #Aptos
Jeffro retweeted
Delighted to announce that JupiterZ is finally live!
With a unique RFQ model designed specifically for Solana, enjoy gasless swaps and 0% slippage with a single toggle 👌
All swaps will pick the best price between our routing engine and RFQ for max convenience to all users.
Jeffro retweeted
绝大多数情况下,没有人会把自己赚到钱的方法告诉你,除非出现下面这三种情况:第一,他自己靠这个方法没有挣到钱,但他要靠售卖这个方法来挣你的钱。第二,他确实靠这个方法挣到钱了,但现在这个风口已经过去了,他告诉你的方法是不是一个很好的方法呢?确实是,但是这不是一个很好的时代,你这个时候入局,最好的结果就是捡别人剩下的,最坏的结果就只能是当韭菜。第三,他确实靠这个方法挣到钱了,并且这个方法现在还能挣到钱,但他不是一般人说做就能做到的,就是即使你知道了这个方法,他也需要你具备很扎实的资源背景,就是门槛很高,他不是谁都能轻易入局的,所以他即使告诉你,对他也构不成什么威胁。
Jeffro retweeted
还能根据
价格,池子大小,价格变化百分比,交易量,最低价,最高价,买家数量,卖家数量,链上交互交易数量等
监控筛选1条链,或者多条链上所有符合条件的土狗,你就说屌不屌吧
Jeffro retweeted
市值前200币种强庄排行榜
第一次作图 轻喷
内容创意取自 @0xjuu_17
Jeffro retweeted
除了 UG 之外的其他机会,这里整理了文档供大家去看,点赞收藏!
docs.google.com/spreadsheets…
个人认为 ANARCHO•CATBU 非常重要,这个名字是Casey一直挂在个人简介上的 symbol,需要重点关注
Jeffro retweeted
A thread 🧵
Some things to keep in mind if you're considering investing in crypto assets:
Jeffro retweeted
Inscriptions are the 700+ hp American muscle cars on the network state super highways
Jeffro retweeted
先别有什么期待,零撸。
不知道怎么个事还没研究我先搞了(转几u的钱到小钱包)
{"p":"src-20","op":"mint","tick":"sols","amt":"1000"}
网站: solinscription.com
dune:dune.com/sols/sols
⚠️注意:用小号连接
Jeffro retweeted
人类智力的最高形式,就是不带情绪的观察。
我们需要把【事件】和【认知】这两件事分开
一个德扑局上大家纷纷讨论自己买的豪宅(事实)
你觉得他们不带你聊天是看不起你(认知)
真正让人类困惑的不是事情本身,而是对事情的看法,看法最终会主导情绪
币安被罚款(事实)->大盘要崩(认知)->恐慌(情绪)->开空单(决策)
女朋友和男同事看电影(事实)->她出轨了(认知)->愤怒(情绪)->提分手(决策)
我们一定要聚焦在【事实】上,确认已经充分观测了整个市场的【认知】和【情绪】,再做【决策】
Jeffro retweeted
集中答疑
仅4个如龙社区活跃成员可上麦
teapawt.net/i/spaces/1mrGmyXXVZQGy
Jeffro retweeted
New paper by Google provides evidence that transformers (GPT, etc) cannot generalize beyond their training data


























