ML Engineer & PhD in NLP
Vancouver, Canada
Joined August 2009
- Tweets152
- Following184
- Followers154
- Likes186
Voice Writer is live on Product Hunt 👍🔼 Check it out and show your support by giving an upvote if you find it useful! producthunt.com/posts/voice-…
Now, with voice technology and and AI for grammar correction, I find I can produce 3-4x more content without worrying about minor language issues — between 2,000 to 3,000 words! 🚀This has allowed me to express my ideas more fully and creatively.
I built a voice writer tool to help you write things quickly. ⚡️ It uses AI for speech recognition and grammar correction. I have been using it for my book reviews, emails, Slack messages, and more. Here is a demo video. 😊
Try it out here: efficientnlp.com/voice-write…
🎥 New Video! In this video, we train a speech recognition model (using OpenAI's Whisper) to recognize our family's Chinese dialect, Teochew, or Chaozhou dialect (潮州话). It has about 10 million speakers and is a part of the Min Nan language family.
youtube.com/watch?v=JH_78KmP…
📹 New Video! Ever had trouble deciding which AI to use for your projects? Let's solve that with AI. In this video, I will build an AI to find the best AI for you 🤯
youtu.be/2r-SqtxhgmY
📹 New Video! #LLMs can be slow, so this @GoogleDeepMind paper proposed to speed it up by running two LLMs at the same time. 😕 Wait what?
It's a new technique called speculative sampling. A smaller LLM generates the easier tokens and a larger LLM checks them. And using a rejection sampling trick, there is no difference in accuracy!
Check out my video on how this works ➡️youtube.com/watch?v=S-8yr_Ri…
📺 New video: Map of Natural Language Processing
youtube.com/watch?v=zF69Eqhi…
Ever since its introduction in 2017, the transformer architecture has remained largely unchanged...until now.
youtube.com/watch?v=o29P0Kpo…
📹 New Video: Understanding the KV Cache
Why is it so difficult to run LLMs with longer context windows? 🤔 The culprit behind this memory hog is none other than the KV cache. In this video, I explain this essential component and its impact on GPU memory during inference.
Watch the video here ➡️ youtu.be/80bIUggRJf4
📈 Don't forget to subscribe and hit that like button if you find this content informative!
In this video, I discuss techniques to boost the speed and compress the size of your AI model for inference: from quantization to pruning, knowledge distillation, and engineering optimizations like GPU acceleration and fused kernels.