助力一下 gopher 今年金九银十跳槽季,小弟我把去年刷的 leetcode 整理出了 520 题汇成了小册子,全部都是 go 实现的 runtime beats 100% 了。在线阅读地址:books.halfrost.com/leetcode/,网页支持了 PWA,可以像 mac 应用一样沉浸式阅读。@chaishushan
@changkun @talkgo_night @unknwon @astaxie
vLLM V1, part 7: Automatic Prefix Caching. Chained block hashes find reusable prefixes; only finalized full blocks enter the cache; the free queue doubles as lazy LRU. github.com/halfrost/Halfrost…
I traced vLLM V1’s KV cache from GPU budget to physical slot: fixed-size blocks, prefix hashes, refcounts, preemption, and block tables. Part 6: github.com/halfrost/Halfrost…
1
524
vLLM V1 has no global “prefill phase” or “decode phase.”
Each step asks: how many tokens may each request process?
Article 05 traces token budgets, RUNNING-before-WAITING, Chunked Prefill, prefix-aware admission & KV-pressure preemption.
github.com/halfrost/Halfrost…
#vLLM
3
8
1,086
vLLM’s heartbeat is EngineCore.step():
schedule → execute → abort barrier → commit
Article 04 traces request states, overlapped batches, optimistic counters, aborts, cleanup & continuous batching—one tick, line by line.
github.com/halfrost/Halfrost…
#vLLM
vLLM V1 is not “an API server calling a GPU.”
It is a process topology:
API server → ZMQ → EngineCore → shared-memory queues → GPU workers.
Article 03 traces GIL isolation, TP/DP math, startup barriers, and worker-death propagation.
github.com/halfrost/Halfrost…
#vLLM
2
5
748
vLLM has 3 front doors: LLM.generate(), vllm serve, and OpenAI-compatible HTTP.
Follow the source and they converge:
process_inputs → EngineCoreRequest → EngineCore.
Article 02 traces offline, CLI, HTTP, SSE & cancellation.
github.com/halfrost/Halfrost…
#vLLM
6
661
What happens between LLM.generate() and the first token returned by vLLM? I traced the complete V1 request path—from input processing and continuous-batch scheduling to GPU execution, sampling, and output processing.
Part 1 of a source-code walkthrough:
github.com/halfrost/Halfrost…
8
564
Replying to @Junlong_x @luoleiorg
这两年 AI 太火,最近我也在做一些关于 AI 的 research,blog 有一段时间没更新了。等作出成果以后,再出来写点文章。
3
604
Replying to @luoleiorg
😅“最终还是活成了自己讨厌的样子”。我承认自己小卷了一下。但是后面的话我兑现了自己的承诺。在 Yosemite 看花开,在 Grand Canyon 看水流,在 Haleakalā 看朝阳升起,在 Half moon bay 看夕阳落下。在 1 号公路上经历了很多有趣的事,在湾区遇见了很多难忘的人。instagram.com/halfrost_/
2
62
3,064





