Conner Takehana retweeted
Last Year We Built Megakernels. This Year We Deleted Them.
One year ago, we helped introduce some of the first AI megakernels to the world. Since then, we’ve enjoyed seeing their wide adoption from friends and researchers. However, while whole model megakernels are a helpful abstraction, we’ve found that they overcomplicate and are not always an ideal optimization target.
Below, we discuss our journey to understand and push past these limitations. We start with a deep dive into two contrasting paradigms of ML optimization – Torch Compile and Megakernels
Two weeks ago we announced Together Compiler: agents that tune inference to your model, chip, and traffic, for up to 2.6x more throughput.
Today, we’re sharing the journey behind it: the experiments that challenged our assumptions of megakernels, the surprising strengths of Torch Compile, and the lessons that shaped our development of a next-generation system for inference optimization.
Shout out to @dylan__lim @reyna_abhyankar @simran_s_arora and the team!
Read more: together.ai/blog/road-to-tog…
Conner Takehana retweeted
introducing chipmunk—a training-free algorithm making ai video generation 3.7x & image gen 1.6x faster! ⚡️
our kernels for column-sparse attention are 9.3x faster than FlashAttention-3 and column-sparse GEMM is 2.5x faster vs. cuBLAS
a thread on the GPU kernel optimizations 🧵
Our latest joint work w/ SandyResearch @ UCSD: training-free acceleration of Diffusion Transformers w/ dynamic sparsity, led by @austinsilveria @SohamGovande!
⚡️ 3.7x faster video and 1.6x faster image generation while preserving quality!
🧵 Open-source code & CUDA kernels!
Happy Thanksgiving! Introducing ThunderMittens for your ThunderKittens, TK kernels for Apple chips! - @AaryanSinghal4 and I
we wrote a blog about TK on M2s: hazyresearch.stanford.edu/bl…
Conner Takehana retweeted
Thankful for TK kernels that now run on Apple hardware - with @CKT_Conner, bringing faster attention and GEMMs to your M2 MacBooks! happy thanksgiving 🍁
Conner Takehana retweeted
Thrilled to be unveiling Thunderkittens 0.1: now with more simplicity, more speed, and more kernels!
Had a blast working on this with @spectorb, @simran_s_arora, and folks at @HazyResearch 🚀
(1/7) In celebration of National Cat Day, we’re excited to release our first major batch of updates to ThunderKittens! ThunderKittens is now easier, better, faster, and cuter than ever before! In addition to massive speed boosts, we’re releasing a broad swath of kernels, new features, and tons of usability improvements.
(Blog post: hazyresearch.stanford.edu/bl…, Paper: arxiv.org/abs/2410.20399)
As always, joint with @AaryanSinghal4, @simran_s_arora, @HazyResearch, and team!
Conner Takehana retweeted
Introducing ambientGPT: an open-source and multimodal MacOS foundation model GUI
Run GPT-4o and open-source models with full ambient knowledge of your screen.
Foundation models have long been confined to the browser. With ambientGPT, your screen context is directly inferred as part of the query, ensuring you never need to explicitly upload context again!
Github: github.com/siddrrsh/ambientG…
Conner Takehana retweeted
The three presidential candidates are now competing to prove that they can pump your bags more than the other two.
We are the new wealthy elite.
Newsy Johnson
Conner Takehana retweeted
Introducing Gemma with a 10M context window
We feature:
• 1250x context length of base Gemma
• Requires less than 32GB of memory
• Infini-attention + activation compression
Check us out on:
• 🤗: tinyurl.com/gemma10hf
• GitHub: tinyurl.com/gemma10git
• Technical blog: tinyurl.com/gemma10blog
Built by: @maxaljadery, @AkshGarg03, @siddrrsh
Conner Takehana retweeted
Price down?
Bullish, it's oversold
Price up?
Bullish, it's trending up
Price flat?
Bullish, it's consolidating
Conner Takehana retweeted
Replying to @blknoiz06
did this test on my wife back in popcats infancy, passed with flying colors, been holding ever since.
Conner Takehana retweeted
.@blknoiz06 meeting people at conferences
Conner Takehana retweeted
Someone just aped 100k into $devin
































