Abhinandan retweeted
hi everyone,
i am an inference engineer, proactively looking for the right team to join.
Things i have done
+ contributed to vLLM, mooncake, dynamo and more
+ have 2.5+ yoe building at startups
+ received direct offer from yc-backed startup through oss
Can specifically help with
+ KV
+ RL rollouts &
+ CUDA kernels
Currently
+ deepening, making more & more contributions to @vllm_project
+ spending ~$25 on Hoppers & Amperes every week
here is my work - abhinandan.one
resume screenshot is attached.
lmk if you/your team/known is looking for a similar person. pls DM me.
if u have read so much, pls like, repost, quote & cfbr 🙏.
will be grateful for ur support.
hi everyone,
i am an inference engineer, proactively looking for the right team to join.
Things i have done
+ contributed to vLLM, mooncake, dynamo and more
+ have 2.5+ yoe building at startups
+ received direct offer from yc-backed startup through oss
Can specifically help with
+ KV
+ RL rollouts &
+ CUDA kernels
Currently
+ deepening, making more & more contributions to @vllm_project
+ spending ~$25 on Hoppers & Amperes every week
here is my work - abhinandan.one
resume screenshot is attached.
lmk if you/your team/known is looking for a similar person. pls DM me.
if u have read so much, pls like, repost, quote & cfbr 🙏.
will be grateful for ur support.
starting the monday with merged PR to @vllm_project /semantic-router
total_merge_count += 1 #38
3
1,095
oss update:
+ 1 PR merged to @nvidia's Dynamo
+ 2 PRs merged to @vllm_project /agetic-api
more the the go....
oss update:
+ 2 PRs merged to @vllm_project /agentic-api
total count +34. more on the go :)
reading was one my favorite habbits, before ai. and i used to the read articles, stories, leetcode solutions and a lot of the things.
and all of them gave interest, fun, knowledge, thrill and so much because i think there were feelings in those readings. i knew that someone human has written those out of emotions and efforts just like talking to a another human or understanding their thinking & emotions.
but lately, i feel my habbit is no longer there. because whenever i start reading, the first things that comes to my mind `ai-generated`. and all the curiosity gets killed at the moment :(
i don't know why but this has become too frequent, and even the genuine content appears to be ai-generated. so generally there is no desire to read new and more.
ai has given us a lot, but somewhere it deprived us that interest, that curiosity of reading other, knowing others, their feeling, their writing, their creativity and a lot of things. i feel there is almost no emotion in reading.
and i think generating content has becomes 100x easier, but finding a genuine readable passage had becomes millions times harder. besides this is my personal thoughts, out of the experiences that i have in past few years.
4
647
ai vs human mind - intellect comparison (ARC-AGI-2).
if we set the avg human at 100, then
astra - 144
6.1 sol - 143
opus 5.5 - 141
gemini 4 Argon - 138
deepSeek V4.1 Flash - 110
grok 4.7 - 93
and this is just the start of ai...
1
6
751
Abhinandan retweeted
EAGLE-4 is here.
From EAGLE-1 to EAGLE-3, we’ve kept making inference faster.
EAGLE-4 is our fastest yet.
961.6 tokens/s on @Alibaba_Qwen’s Qwen3.8-27B with TokenSpeed (BF16, 4× @NVIDIA B300, TP4, one request, T=0).
Github: github.com/eagle-hq/EAGLE-4
Abhinandan retweeted
deepseek has given a lot
+ the first* oss reasoning model (the viral launch of R1, jan 2025)
+ MLA
+ DSA
+ MTP (spec-decoding)
+ FlashMLA
+ DeepGEMM
+ DeepEP
+ EPLB
+ 3FS
+ @DeepSeekHarness
+ DeepInfer
and its all open-source.
thank u so much @deepseek_ai for this all 🙏
🚀 Introducing DeepInfer, our in-house inference framework, now fully open source!
Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable.
🔗 github.com/deepseek-ai/DeepI…
1/4
deepseek has given a lot
+ the first* oss reasoning model (the viral launch of R1, jan 2025)
+ MLA
+ DSA
+ MTP (spec-decoding)
+ FlashMLA
+ DeepGEMM
+ DeepEP
+ EPLB
+ 3FS
+ @DeepSeekHarness
+ DeepInfer
and its all open-source.
thank u so much @deepseek_ai for this all 🙏
🚀 Introducing DeepInfer, our in-house inference framework, now fully open source!
Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable.
🔗 github.com/deepseek-ai/DeepI…
1/4
Abhinandan retweeted
🚀 Introducing DeepInfer, our in-house inference framework, now fully open source!
Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable.
🔗 github.com/deepseek-ai/DeepI…
1/4
Abhinandan retweeted
Today, Dream Labs is coming out of stealth with Dream-1, a robot foundation model that controls bimanual robots it has never seen, on tasks it was never shown, in places it has never been, from $3/hour.
Dream API is live for everyone, today. 🧵
oss update:
+ 2 PRs merged to @vllm_project /agentic-api
total count +34. more on the go :)
oss update:
+ 1 PR merged to @KVCache_AI/ Mooncake
+ 1 PR merge to @vllm_project /semantic-router
more on the go... 🚀
Abhinandan retweeted
Spent years watching models ace a single image and completely whiff on anything that moves or changes angle. Today we’re fixing that.
Try Glimpse now, our first visual thinking model with competitive frontier performance compared to Astra on key public visual reasoning benchmarks.
congrats @vllm_project 💙
In 2022, Berkeley PhD student Woosuk Kwon thought he could make Meta's new open-weight model, OPT-175B, run faster in two weeks.
Four years and 3,000 contributors later, that project is @vllm_project, one of the largest open-source inference engines, running on ~500,000 GPUs at any given time.
He and fellow researchers Simon Mo, Kaichao You, and Roger Wang borrowed "virtual memory" from operating systems, handing out memory only as it's needed so each GPU can juggle requests of wildly different sizes. (The "v" in vLLM is for virtual.)
Their first user, Berkeley neighbor @arena, cut its GPU needs in half. Today Google, Amazon, and Meta all run vLLM in production.
In January, the core maintainers founded @inferact and raised $150M to give vLLM a full-time team.
They're hiring across eng, marketing, and devrel in SF and Singapore: a16zjobs.substack.com/p/infe…
@simon_mo_ @woosuk_k @KaichaoYou @rogerw0108
5
286
how do u handle the async rollout and cache invalidation when weights change in vllm.
so experimented with it. built rolloutcore
and wrote and an artifact on it
abhinandan.one/artifacts/bui…
2
4
290
oss update:
+ 1 PR merged to @KVCache_AI/ Mooncake
+ 1 PR merge to @vllm_project /semantic-router
more on the go... 🚀
oss update:
+ 1 PR merged to @nvidia 's dynamo
+ 1 PR merged to @vllm_project /vllm-ascend
more on the go.... 🚀

















