Abhinandan (@abhibuilds) header
Abhinandan (@abhibuilds) profile picture

inference โ€ข research โ€ข rl

cuda
Joined December 2025
Photo by Abhinandan (@abhibuilds)Photo by Abhinandan (@abhibuilds)Photo by Abhinandan (@abhibuilds)Photo by Abhinandan (@abhibuilds)Photo by Abhinandan (@abhibuilds)Photo by Abhinandan (@abhibuilds)Photo by Abhinandan (@abhibuilds)Photo by Abhinandan (@abhibuilds)Photo by Abhinandan (@abhibuilds)Photo by Abhinandan (@abhibuilds)
Abhinandan retweeted
hi everyone, i am an inference engineer, proactively looking for the right team to join. Things i have done + contributed to vLLM, mooncake, dynamo and more + have 2.5+ yoe building at startups + received direct offer from yc-backed startup through oss Can specifically help with + KV + RL rollouts & + CUDA kernels Currently + deepening, making more & more contributions to @vllm_project + spending ~$25 on Hoppers & Amperes every week here is my work - abhinandan.one resume screenshot is attached. lmk if you/your team/known is looking for a similar person. pls DM me. if u have read so much, pls like, repost, quote & cfbr ๐Ÿ™. will be grateful for ur support.
52
37
2
796
67,256
Abhinandan retweeted
EAGLE-4 is here. From EAGLE-1 to EAGLE-3, weโ€™ve kept making inference faster. EAGLE-4 is our fastest yet. 961.6 tokens/s on @Alibaba_Qwenโ€™s Qwen3.8-27B with TokenSpeed (BF16, 4ร— @NVIDIA B300, TP4, one request, T=0). Github: github.com/eagle-hq/EAGLE-4
932
677
245
11,466
5,229,867
Abhinandan retweeted
deepseek has given a lot + the first* oss reasoning model (the viral launch of R1, jan 2025) + MLA + DSA + MTP (spec-decoding) + FlashMLA + DeepGEMM + DeepEP + EPLB + 3FS + @DeepSeekHarness + DeepInfer and its all open-source. thank u so much @deepseek_ai for this all ๐Ÿ™
๐Ÿš€ Introducing DeepInfer, our in-house inference framework, now fully open source! Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. Weโ€™re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable. ๐Ÿ”— github.com/deepseek-ai/DeepIโ€ฆ 1/4
1
3
9
549
Abhinandan retweeted
๐Ÿš€ Introducing DeepInfer, our in-house inference framework, now fully open source! Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. Weโ€™re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable. ๐Ÿ”— github.com/deepseek-ai/DeepIโ€ฆ 1/4
1,768
4,938
2,352
43,866
7,123,812
Abhinandan retweeted
Today, Dream Labs is coming out of stealth with Dream-1, a robot foundation model that controls bimanual robots it has never seen, on tasks it was never shown, in places it has never been, from $3/hour. Dream API is live for everyone, today. ๐Ÿงต
153
185
166
1,670
632,019
Abhinandan retweeted
Spent years watching models ace a single image and completely whiff on anything that moves or changes angle. Today weโ€™re fixing that. Try Glimpse now, our first visual thinking model with competitive frontier performance compared to Astra on key public visual reasoning benchmarks.
408
646
427
6,099
4,296,380
Abhinandan retweeted
Attention is the first step in understanding the models. But generally, it is not easy to understand it in one definition or directly jumping to QKV & softmax. So I tried to make it easy and simple to understand it from the first principles intuition.
2
2
1
10
1,096
Abhinandan retweeted
oss update: + 1 PR merged to @nvidia 's dynamo + 1 PR merged to @vllm_project /vllm-ascend more on the go.... ๐Ÿš€
oss update: + 7 PRs merged to @vllm_project /semantic-router + 1 PR merged to @KVCache_AI /Mooncake
1
2
1
8
836
Abhinandan retweeted
oss update: + 7 PRs merged to @vllm_project /semantic-router + 1 PR merged to @KVCache_AI /Mooncake
small oss update: 4 prs merged to @vllm_project /semantic-router
2
1
12
6,808
Abhinandan retweeted
i am an inference engineering unit, aggressively looking for a relevant great team to join. i have + been digging deep into transformers, scheduling, caching, quantization every new day + been continuously contributing to @vllm_project , @nvidia's TensorRT-LLM, dynamo, @KVCache_AI 's mooncake, @sgl_project , @_llm_d_ and more + built projects on scheduling, batching, KV & prefix caching, quantization, HiCache, RL rollouts + paying 20+ dollars to @runpod almost every week besides + been founding engineer at fast-paced ai startups, overall 2.5+ yoe + received direct offer from a yc-backed startup through open-source contributions + been recognised as top 1% typeScript engineer globally on Algora and > have built cdp-native browser-use agents > auto demo video generation system using @Remotion > proactive DevRel system for content/docs > process graphs for hr automation workflows plus ^ i work 12+ hours almost everyday ^ generally 12pm to 4am ^ i work in silent room with no headphone open to > remote > relocation > onsite > give you provide enough value no - girlfriend - parties - smartphone distraction( avg 12 mins/day) โ†’ here is my work - abhinandan.one โ†’ github - githhub.com/awesome-pro if you feel that I can create a worth value for your team - lmk, i am available. DMs are open or if someone you know might need me, please refer - will be highly grateful and if you have read so long, please like, quote/repost, cfbr - your small help may give me reach my next team. finally thanks for reading.
Which startups are actively hiring? (September thread) Reply with company name, career page, location
1
2
18
1,175
getting acquired by @nvidia = hugging face can now hire people we couldn't as a small startup and give them a decade to make open-source AI win! if you're one of them, my dms are open
272
319
36
7,066
378,258
Abhinandan retweeted
got my first contribution merged to @vllm_project so happy - nothing to say more.
1
1
5
316
Abhinandan retweeted
Reasoning from scratch, round number 5! This time, talking about log-probability scoring (also a great fundamental concept for loss functions like cross-entropy in pre-training and distillation) and self-refinement. 00:00 Introduction and inference-time scaling recap 05:02 Loading the pretrained LLM 08:00 Comparing and scoring model answers 10:18 Building a rule-based scorer 17:53 Token probabilities and sequence likelihood 26:47 Computing token probabilities in PyTorch 30:12 Token indexing and shifted targets 37:27 Log probabilities and numerical stability 45:57 Scoring answers with average log probabilities 56:24 How self-refinement works 59:07 Generating critiques and revised answers 1:01:00 Implementing the self-refinement loop 1:05:57 MATH-500 evaluation results 1:07:35 Takeaways and next steps
43
130
5
1,030
61,096
Abhinandan retweeted
Wake up, ppl ๐Ÿ‘€ Xiaomi just open-sourced ~7K of the RL environments it used to train MiMo on HuggingFace, across domains like code, cyber, general, music and web dev I think this is one of the biggest things to happen in frontier open-source RL environment data. Indepth analysis and learnings coming soon ๐Ÿ‘€
43
153
19
1,691
189,432
Abhinandan retweeted
here is an overview of the 4 most advanced efficient architectures: Deepseek V4.1 Flash, MiMo V3, Qwen 3.8 Next Flash and GLM 5.3 Flash Deepseek and MiMo are quite similar, they both use YOCO - only the first part of the network is active during prefill to build the KV cache representation - no linear attention, and token level indexer Qwen and GLM both use a more standard interleaving 3:1 (like Kimi K3 as well) between sparse attention and linear attention (GDN vs KDA) both deepseek and qwen use Engram, they all use a gate or sink except GLM 5.3 Flash, they also all use no or partial RoPE on the full/sparse attention layers they also all have some sophisticated residual network, either mHC (simplified or not) or gated residual and they are all trained using Muon visualization by opus 5.5 and me :)
19
80
1
745
28,567
Abhinandan retweeted
Why do LLMs not generate same response every time, for the exact same prompt provided. The reason is sampling. Sampling makes the output generation probablistic unlike the Greedy generation. Temperature determines the entropy or creativity of the output tokens. While topK and topP control the selectivity of the tokens. So in this video I have explained how does the sampling based generation work. How the temperature, topK & topP matter in this generation of output. Everything from first principles in easy to understand way. This is the Part 2 of Building an LLM Inference Engine from Scratch - Sampling Inference Generation Complete Video & Logic Code in first reply
What happens between your prompt and the tokens you see on screen? ๐—”๐—ป ๐—Ÿ๐—Ÿ๐—  ๐—ฑ๐—ผ๐—ฒ๐˜€๐—ป'๐˜ ๐—ด๐—ฒ๐—ป๐—ฒ๐—ฟ๐—ฎ๐˜๐—ฒ ๐—ฎ๐—ป ๐—ฎ๐—ป๐˜€๐˜„๐—ฒ๐—ฟ ๐—ฎ๐—น๐—น ๐—ฎ๐˜ ๐—ผ๐—ป๐—ฐ๐—ฒ. It predicts one token, appends that token to the sequence, runs the model again, predicts the next token. And keeps repeating this process. Most of us know this concept, but I wanted to understand what the inference loop actually looks like I build it myself. So I have started building a mini LLM inference engine from scratch. In this Part 1, I implemented greedy autoregressive generation: input_ids [B,T] โ†’ model forward pass โ†’ logits [B,T,V] โ†’ take the last position [B,V] โ†’ choose the highest-scoring token โ†’ append it โ†’ repeat Building such a generation loop, makes you to properly understand - what logits represent, - why the output has shape [B,T,V], - why generation only uses the last position, - and how the model's own prediction becomes the input for the next step. All this question I have tried to answer in this video in easy to understand way. This is the first part of a series Building Inference Engine where I'm progressively adding: ๐˜€๐—ฎ๐—บ๐—ฝ๐—น๐—ถ๐—ป๐—ด โ†’ ๐—ž๐—ฉ ๐—ฐ๐—ฎ๐—ฐ๐—ต๐—ฒ โ†’ ๐—ฏ๐—ฎ๐˜๐—ฐ๐—ต๐—ถ๐—ป๐—ด โ†’ ๐—ฝ๐—ฎ๐—ฑ๐—ฑ๐—ถ๐—ป๐—ด/๐—บ๐—ฎ๐˜€๐—ธ๐˜€ โ†’ ๐—ง๐—ง๐—™๐—ง/๐—œ๐—ง๐—Ÿ
1
1
1
237