Pinned Tweet
🚨 New paper! SHANKS lets spoken language models (SLMs) think while listening💭👂
This enables the SLM to interrupt the user in a timely manner and make early tool calls when the speaker is still speaking.
Paper: arxiv.org/abs/2510.06917
Project page: d223302.github.io/SHANKS/
Check out our latest paper on instruction-aware speech retriever, IRIS. It can retrieve a speech/audio based on a spoken query and a text instruction! Our analysis also shows that the intermediate layers from the audio encoder carry a lot of information in the audio and are useful for searching for audio based on diverse instructions
🚨 New paper alert! 🎙️
From INSPIRE to IRIS-Embed!
Can speech retrieval understand what users want to find, beyond what was said?
Excited to share IRIS-Embed, our new approach to instruction-aware speech embeddings. 🧵
📄 arxiv.org/abs/2611.09122
7
243
Cheng Han Chiang (姜成翰) retweeted
📢 Please note a critical update to the original #EMNLP2026 Call for Papers: since EMNLP and AACL share the ARR May cycle, authors will need to explicitly select a target conference at submission time. This choice will be binding!
For more details see: 2026.emnlp.org/calls/main_co…
📢 The First Call for Papers for EMNLP 2026 is officially out! 📝
We welcome long & short papers featuring original research on empirical methods for NLP.
🗓️ ARR Submission Deadline: May 25, 2026
🔗 Read the full CFP here: 2026.emnlp.org/calls/main_co…
#EMNLP2026
Glad to see that all four reviewers I spent a lot of time writing for are classified as full human-written. Kind of
ICLR authors, want to check if your reviews are likely AI generated?
ICLR reviewers, want to check if your paper is likely AI generated?
Here are AI detection results for every ICLR paper and review from @pangram!
It seems that ~21% of reviews may be AI?
5
527
Cheng Han Chiang (姜成翰) retweeted
LLM reasoning/tool use improves results but adds latency, hindering real-time dialogue. SHANKS is a new method that enables simultaneous "hearing" and "reasoning/tool use" to reduce this lag. Work from Cheng-Han Chiang & Microsoft researchers. arxiv.org/abs/2510.06917
🚨 New paper! SHANKS lets spoken language models (SLMs) think while listening💭👂
This enables the SLM to interrupt the user in a timely manner and make early tool calls when the speaker is still speaking.
Paper: arxiv.org/abs/2510.06917
Project page: d223302.github.io/SHANKS/
This work was done during my internship at Microsoft with @Orpheus_wang, @LINJIEFUN, Chung-Ching Lin, @linkeyun2, Shujie Liu, @ZhendongWang6,
@zhengyuan_yang, @HungyiLee2, Lijuan Wang
1
1
192
If you are interested in thinking and reasoning for SLM, our previous work, STITCH, is a must-see.
STITCH enables SLMs to think while speaking.
Paper: arxiv.org/abs/2507.15375
Demo: d223302.github.io/STITCH/
1
172
Cheng Han Chiang (姜成翰) retweeted
SPS SLTC/AASP Webinar: Foundational Speech Models and their Efficient Training with NVIDIA NeMo
Date: 27-August-2025
Time: 9:00 AM ET (New York Time)
Presenter: Dr. Piotr Żelasko
Registration link: landing.signalprocessingsoci…
🎉 Excited to share that our paper on audio-LLM-as-a-judge has been accepted to EMNLP 2025 Findings!
🔗 arxiv.org/abs/2506.05984audi…
🗝️ Highlights:
🧑⚖️ Agreement between human and audio-LLM-judge can be as high as human-human agreements
👑 Gemini-2.5-pro outperforms GPT-4o-audio as a speaking-style judge
🗣️ There's still room for improvement in style following & natural dialogue generation for SLMs
Our work shows strong potential for using audio LLMs to evaluate speech generated by spoken language models (SLMs) 🎙️✨
Excited to see future work explore fine-tuning SLMs with rewards or feedback from these audio-LLM judges 🧑⚖️🔁
1
3
319
This work was done during my internship at Microsoft GenAI, mentored by @Orpheus_wang, collaborating with Chung-Ching Lin, @linkeyun2, @LINJIEFUN, Radu Kopetz, Yao Qian, @ZhendongWang6 , @zhengyuan_yang , @HungyiLee2 , Lijuan Wang
Thanks for the great mentorship and collaboration.
2
285
Cheng Han Chiang (姜成翰) retweeted
This work began as my summer intern project last year, and has since grown into a long-term effort in large-scale pretraining.
Huge thanks to project leaders @Hu_Hsu @ShangwenLi1, and @yangli625 @dongwang218 for all the dedication!
@metaai @MIT_CSAIL
🧑💻 github.com/facebookresearch/…
Excited to be at #ACL2025! 🎉
Looking forward to sharing our research and learning from the community. Open to discussions on:
- LLM-as-a-judge: our paper, TRACT, was presented this morning
- Audio-LLM-as-a-judge for speaking style
- STITCH: Our newest SLM that can think and talk simultaneously
Let’s connect! 🤝
#ACL2025NLP
1/7 🔗 Introducing STITCH: our new method to make Spoken Language Models (SLMs) think and talk at the same time.
Paper link 👉 arxiv.org/abs/2507.15375
16
610
1/7 🔗 Introducing STITCH: our new method to make Spoken Language Models (SLMs) think and talk at the same time.
Paper link 👉 arxiv.org/abs/2507.15375
6/7 🆕 We explore another variant called STITCH‑S, where the model generates a spoken response chunk before the first reasoning chunk.
🏃 STITCH‑S has the same latency as a non‑reasoning model by design but delivers much better performance.
1
269












