罗杰斯 retweeted
le french saw h-net and decided to tax bytes instead of tokens
So, the French socialists, and Parliament members just voted a tax, that they don’t understand, which would cost VideoLAN more than 1 million euro per year.
They are probably the most clueless politician on Earth? And they claim they want technological sovereignty…
@LaurentLhardit
罗杰斯 retweeted
Didn’t see anyone mention this so guess I will
My team and I looked into H-Nets pretty seriously when we first started exploring LLM efficiency
It’s genuinely super interesting, and what they’ve done is very impressive, but I’m still skeptical of the inference economics at scale
Dynamic chunking means you deal with unpredictable compute per request, which hence means harder batching and more complicated GPU scheduling/memory management
But there are ways to handle this by using separate queues and grouping requests by expected workloads + custom kernels, but i don’t think it’s sufficient to beat the competitions
A hybrid architecture might be interesting though!
Because you can reduce FLOPs on paper and still end up with worse throughput, higher latency and more expensive inference in prod
At the end of the day what really matters is how many requests you can serve per GPU at the same quality and latency
But again
That really doesn’t take anything from what @_albertgu and his team have achieved, we gave up on H-nets due to the above and a few other reasons but regardless, it still has very significant use cases even if it struggles to scale
Gg
I've always believed that intelligence should come from simple, general building blocks that learn their own abstractions directly from raw data; in other words, the bitter lesson. I think we're finally seeing a real step towards AI that can learn from all data.
Language modeling on raw bytes is a microcosm of this ideal, and to me it's a central problem in AI: if we can't build LLMs that learn end-to-end from bytes, how will we handle the rest of the world's data that we understand even less?
Last year we introduced hierarchical networks (H-Nets) as a first step. Our new version finds up to 3-5x efficiency gains in controlled scaling law experiments, and at larger scale matches an open-source model trained on 11T tokens with ~10x less data and compute. The posts below walk through what we found and why we think it works.
罗杰斯 retweeted
What I find annoying is that everyone celebrates how cheap Engram is but nobody wants to ask "is Engram any good tho?" Like, you're not getting a free offload of FFNs to DRAM. How much does it add to modeling?
I trust DeepSeek, but that's me. Does… everyone just trust them?
🫡梁文峰在记忆层动了手脚！为什么bDeepSeek V4.1 Flash 又便宜又强？！不是堆了更多显卡，是把“背课文”从 GPU 里拆走了！
V4.1 Flash 的 Engram 把常见词组变成查表，不该算的不再占显存。
SemiAnalysis 拆完记忆层，卡上的数字更刺眼：
🔹输入只激活 80 亿、输出 160 亿，剩下约 1960 亿是查找表
🔹浅层记物体，深层记关系，熟词组直接查，推理留给模型
🔹表只看 token ID，能扔进内存；B300 上从 TP4 降到 TP2，性价比最高抬约 1.6 倍
🔹4 张 GB300 把表卸到主机后，KV 容量还能再涨约 36%
🔹B200 上内存卸载打赢 SSD：同等体验附近，每美元总 token 约 1.21 亿对 5200 万
🔹双机 GB300 上，预填充能到约 5.6 万 token/秒，单用户解码约 253 token/秒
便宜来自少算、少占 HBM；
强来自该查就查、该算就算。
显存带宽比显存容量更值钱。
#DeepSeek #V41Flash #Engram #HBM #B300 #GB300 #SemiAnalysis #梁文峰 #梁圣
罗杰斯 retweeted
wow, that's actually very careful designed architecture.
Most of the CSA Full layers and Engram are in the encoder phase, and mostly the decoder uses indexer share modules.
Using mostly encoder outputs as KV cache, which constitutes to 50% of layers makes it run faster. This is super smart!
罗杰斯 retweeted
Holy shit have y'all seen Anthropic ARR?
As measured by last 1 hour at 2PM times 8760 – at San Francisco, CA
罗杰斯 retweeted
This isn't a chip. It's a video of a wooden box getting forklifted combined with name dropping and a narrative about some alleged chip without any details.
罗杰斯 retweeted
Introducing GLM-5.3: Built to Code. Ready for Cyber Defense.
- Top-tier coding and agentic capabilities, achieved through post-training on the 743B base model
- A major leap in cybersecurity, setting a new standard among open models
Tech Blog: z.ai/blog/glm-5.3
罗杰斯 retweeted
am i insane or is that a straight motherfucking line
Gemini 3.7 Flash (High) has reshaped the Pareto frontier in the Code Arena: WebDev and Text Arena!
With a score of 1588 pts in the Code Arena: WebDev, it's priced at $0.75/ $3.75 per MToken. See Text Arena in the post below.
Congrats to the @GoogleDeepMind team!
罗杰斯 retweeted
Hugging Face is so cooked teapawt.net/OpenAI/status/20879477…
罗杰斯 retweeted
It used to be the case that reviewing a contributor's PR was a form of "investment" in that contributor. They'd learn from your feedback, grow, take it into account for future contributions, etc.
Unfortunately with agents I feel that's largely gone.
