Tim retweeted
Stuck at a machine? Send a photo. Get the next step.
AccessPoint guides older, blind and migrant users through ticket machines and kiosks, in their language. It stops wrong totals and never needs your PIN.
Built on @AIatMeta Muse Spark and SAM 3.1.
accesspoint-bot.aljabrialam.…
you'll know more about inference than 90% of people if you fully understand this article
this is only the first resource in the ai performance engineering repo btw.
imagine the ball knowledge in the other ones
we launched the most comprehensive ai performance engineering repo in the world last week
now we'll be posting every single resource
this is Wafer's ai performance engineering series
save this as your starting point. links in thread 🧵
part 1: "All About Transformer Inference" from How To Scale Your Model.
- the authors cover the computations, memory traffic, and serving decisions behind transformer inference:
- arithmetic intensity of linear layers and attention across prefill and decode.
- kv cache sizing by layer count, kv heads, head dimension, sequence length, and precision.
- the compute/hbm bandwidth crossover and how batch size and quantization shift it.
- decode latency and throughput bounds from parameter bytes, kv bytes, and hardware bandwidth.
- weight reuse through batching and diminishing throughput gains as kv traffic grows.
- gqa, kv quantization, and PagedAttention, including the memory costs each addresses.
- model sharding, kv placement, and collective communication overhead.
- continuous batching, prefix caching, and disaggregated prefill/decode.
the worked problems connect model dimensions to deployment decisions like memory capacity, workload distribution, and expected performance under the stated assumptions.
Tim retweeted
【9/9ハッカソン開催のおしらせ】
CODAS×Tavily×Nebius共同主催!
AIプロジェクトに挑戦しませんか?
経験・チーム・事前のアイデアは不要。
🗓 9/9(水) 16:00〜21:00
📍東京23区内(詳細は参加者にご連絡)
🎟無料
申し込みはこちら👇
luma.com/qa0c59fy
Tim retweeted
There are still some spots in the room, but they are running low fast!
starting soon, in hall 4 #3
see you all there and follow that thread :)
Tim retweeted
It’s been ages since I shipped code myself—I felt like a COBOL dev lost in the React age. Then vibe coding hit, and I had to try it firsthand. Welcome to ImageBattle ⚔️
Tim retweeted
Our November Papers of the Month is now live.
This edition covers 4 papers, on "super weights", context-parallelism, scaling laws for precision and critical batch sizes. We provide our summaries and analysis of each. (🧵1/n)
graphcore-research.github.io…
Tim retweeted
1/ Can AI scaling continue through 2030?
We examine whether constraints on power, chip manufacturing, training data, or data center latencies might hinder AI growth. Our analysis suggests that AI scaling can likely continue its current trend through 2030.
Tim retweeted
Andrej Karpathy talking about -
- rarity of talents in distributed computing with GPUs.
- the opportunities in computer architecture because von Neumann architecture may not be optimal
- progress in precision
- sparsity in neural networks
Tim retweeted
The recording of our talk at @letsunifyai's reading group is now available: youtube.com/watch?v=xq_8dgQF…
Thanks to Unify for having us. It was an excellent session with some great discussion
Really excited to announce @luka_ribar and @savelichic from @graphcoreai as our guests for today's live reading group! 😊They will be presenting "SparQ Attention: Bandwidth-Efficient LLM Inference", which is a technique for increasing the inference throughput of LLMs by reducing the memory bandwidth requirements within the attention blocks 🔥
If you'd like to learn more, then join our meeting room today at 7:00 am PT, 10:00 am ET, 2:00 pm GMT, 3:00 pm CET, 7:30 pm IST ✨ Link below ⬇️
All are welcome, from beginners 🧑💻 to experts 🥷
See you there ✌
Meeting: meet.google.com/qsb-phmd-yxi
Livetream: youtube.com/live/xq_8dgQF5Dk…
Paper: arxiv.org/abs/2312.04985
GitHub: github.com/graphcore-researc…
Tim retweeted
Bringing Large Language Model power to the people! Pienso + Graphcore scoop Best Innovation in NLP at the CogX Awards. #CogXAwards23 #CogX_Festival
hubs.la/Q0220RXF0
Tim retweeted
The Llama has landed! Meta's open-source LLM Llama-2 is now available to run - for free - using Paperspace Gradient Notebooks, powered by Graphcore IPUs.
hubs.la/Q01ZBWQX0
Tim retweeted
Further expansion of the Graphcore software ecosystem: UbiOps adds IPU support, taking care of serving, workflow management and dynamic scaling of companies' AI workloads. @UbiOps_ @gcore_official
hubs.la/Q01YYz0z0
Tim retweeted
Want to level up your AI Startup?
Apply for the @graphcoreai #AIStartup Programme. You could receive benefits like:
✔️ Up to $200K IPU cloud credits
✔️ Dedicated #ML expert support
✔️ Bespoke training & hands-on developer workshops
Learn more 👉 hubs.la/Q01XfWpF0
Tim retweeted
🔥 Epic lineup for #GenerativeAIMeetup London 👇
- @nathanbenaich on the #genai landscape
- @cloud_nlp on gen ai in production
- @hellopaperspace on GTM for gen ai #startups
- @graphcoreai on advanced compute for gen ai
Sign up ➡️ hubs.la/Q01QlNHY0
Tim retweeted
Check out the #GenerativeAIMeetup lineup here:
- @nathanbenaich on the #genai landscape
- @juliensalinas on gen ai in production
- @hellopaperspace on GTM for gen ai #startups
- @graphcoreai on advanced compute for gen ai
graphcore.ai/generative-ai-m…
Tim retweeted
My weekend project is now available:
typingmind.com
I was so annoyed by how ChatGPT slowly typed out the answers + kept logging me out every day, so I built a new UI on top of its API.
(plus some more convenience features)
Free to use with your own OpenAI API key 😄
Tim retweeted
The Graphcore community channel on Slack is now open to everyone - a forum for discussion, help, advice and the latest Graphcore news.
hubs.la/Q01BXS_n0
Tim retweeted
Try our OGB-winning GPS++ model yourself for free on @HelloPaperspace
1-click Notebook for training: hubs.la/Q01vrspl0
1-click Notebook for inference: hubs.la/Q01vrpJH0
@NeurIPSconf #NeurIPS #OGBLSC #GNN @dominic_masters























