Founder, hacker, stochastic tool. @microsoft escapee. Currently @eigenlabs. Checkout my github actions emulator: https://xitter.cf/t.co/TPIOeqH6U7
United States
Joined September 2018
- Tweets4.5K
- Following709
- Followers1.5K
- Likes3.1K
Pinned Tweet
I got DeepSeek V4.1 Flash running on a M5 Max with 128gb faster than Astra in Codex. Its running nearly at 40tps at 128k sizes and below.
The model is a repack of @MiaAI_lab's ExLlamaV3 for the 2x Spark configuration with binary contiguous experts to assist in paging data into memory efficiently.
huggingface.co/OpensourceWTF…
I've been quiet because I've been busy with porting my streaming stack to mlx-serve as a mlx-stream plugin. It is designed to load expert layers predictively. Its also using typical acceptance tuned to .3 to get about a 20% speed improvement.
Its not intended that this model is generally usable but its intended that this streaming plugin is used for research. We are keeping slightly less than half of experts in memory in DSv4.1 Flash but combinations of future models and devices will inevitably produce a good model where we can keep most of the experts in memory.
Streaming works like a logistic curve so the higher % of a model loaded into memory, the closer it will run to its theoretical ideal speed. Parts of the curve are very steep, if we can stay entirely above the steep part of the curve, then performance will stay very good or fall off a cliff completely (128k context as can be seen below).
Being able to characterize this for various models is important for research and enabling us to get this working for really good slightly too large models.
TheDavidTai retweeted
Here we go again. Some AI red team startup found and exploited a vulnerability to access a bunch of sensitive documents at Anthropic.
Got access via some third party SaaS Anthropic uses.
TheDavidTai retweeted
BREAKING: Anthropic officially files its S-1 registration statement for an IPO, seeking to list on the Nasdaq under the ticker symbol $ANT.
FULL SUMMARY OF ANTHROPIC'S S-1 FILING:
1. Anthropic posted H1 2026 revenue of $16.3 billion with a net loss of $80.7 billion
2. Anthropic says it does not expect to pay a dividend "in the foreseeable future"
3. Expects to spend up to $120.5 billion on SpaceXAI and Nvidia compute
4. Expects to spend up to $114.5 billion with Google through 2033
5. Revenue growth rate "may not be sustainable" and "may decrease"
6. May be required to seek additional financing "sooner than expected"
7. 5% of Anthropic's shares will go to the "Anthropic Public Benefit Fund"
8. S-1 includes warnings of AI's "catastrophic existential risks to humanity"
Markets currently expect Anthropic to raise as much as $100 billion at a valuation of ~$2 trillion.
The company is reportedly targeting an IPO in the week of November 16th.
TheDavidTai retweeted
IBM's Granite 5 has been hiding in plain sight for 30 days 🪨
Since Sept 25, a public SGLang pull request from an IBM engineer has listed accuracy tests for two unreleased "granite-5.0 MoE" models:
• 20B: 56 experts, hidden size 2048
• 120B: 128 experts, hidden size 3072
• Both run in 4-bit (MXFP4, plus NVFP4 on the 20B) on H100s
No one has reported it. IBM hasn't announced Granite 5 language models; only Granite Speech 5.0 is out.
TheDavidTai retweeted
It's a much bigger jump than previously thought after some more tweaks. 21% increase without spec decode, 32% with spec.
These numbers are averages after running 10 requests (sequentially) in 3 separate servers to minimize any variance.
New all-reduce kernels that optimize for small-message latency across PCIe-connected GPUs. 12% end-to-end decode speedup for DeepSeek-V4.1 Flash on 4x RTX 6000 Pro Server GPUs.
github.com/dphnAI/sonar/pull…
We're basically building cheat engine for LLMs.
I built a tool to control your models refusals from the KV cache without modifying the weights. Handy for Cyber capable models
Phantom-KV:
Refusal removal as a loadable KV-cache graft zero weight modification, no refusal-direction projection, fully reversible.
github.com/lordx64/phantom-k…
To be fair, elections, rate hikes, and end of year stuff make financing hard this time of year so its not really abnormal.
Pretty much.
As of the past few weeks, debt markets have almost entirely shut off access to money for any company that doesn’t have substantial revenue independent of AI data centers.
SpaceX closed $40B of debt for GPUs, but we’re investment grade and have Starlink & launch revenue.
That means the 200+ neoclouds are going to see guppy-level death and consolidation. Some will make it through, but not many.
TheDavidTai retweeted
1/7 🥥 New speculative decoding method!
Introducing DCOCO: Contextual Correction of Parallel Drafts.
Parallel drafters are fast, but lose context between predicted tokens.
DCOCO restores it without giving up parallelism.
Up to 6.6× faster than vanilla decoding. 🧵
TheDavidTai retweeted
▲ Hugging Face just released Carbon-X, an open 14.8B model that reads raw DNA inside a chat and answers in English.
The key idea is 1 decoder for both DNA and text. A hybrid tokenizer keeps Qwen3's text vocabulary and turns DNA into 6-base tokens. There is no separate DNA encoder: it starts from Qwen3-14B, then gets 302B tokens of DNA + text mid-training, SFT and RL.
Across 5 DNA benchmarks it averages 66.7% with thinking off, vs 67.5% for GPT-6.1 Sol at max effort. It leads every model tested on BioReason Coding (74.9%) and NextBase (39.4%). It is a trade-off though: splice-site questions score only 29.8%, and it is not clinically validated.
Weights are Apache 2.0 and fit on 1 x 80 GB GPU. Hugging Face estimates about $300 to score 2.4M ClinVar variants, vs $32K to $4M on frontier models......
Full analysis: marktechpost.com/2026/10/25/…
Model: huggingface.co/HuggingFaceBi…
Blog: huggingface.co/spaces/Huggin…
Repo: github.com/huggingface/carbo…
GGUF: huggingface.co/HuggingFaceBi…
@huggingface @_lewtun @LoubnaBenAllal1 @ClementDelangue
TheDavidTai retweeted
I don't take money from any lab. I'm saying this impartially; the number of people on my feed over the last two weeks saying they are switching over to Anth because of Opus 5.5 is like nothing I've seen before. And even if someone does beat Opus, they have Fable 5.5 ready to go.
Turns out Ant's tiny llm on a TV test was already a thing.
Ok, I just checked this, and its literally running a freaking TensorFlow Lite model on 960×540 images with INT8 quantization every couple of seconds in an attempt to figure out what game you are playing.
It's literally called "alex.tflite" and its literally AlexNet and seems to follow torchvision.models.alexnet. As far as I can tell this does some strange nearest neighbor feature comparisons against a DB and literally doesn't work 99% of the time, you love to see your CPU get wasted every other second by this thing though. It also just seems to dump its results into the void because the consent gate comes later.
Anyways, here's the tf model graph and some dumped captures.
Disable your tailscale ssh if you use it (I've already recommended to not use it).
‼️ Researcher Erica Windisch skipped coordinated vulnerability disclosure and published 11 unpatched Tailscale flaws.
In her lab, Tailscale SSH race conditions let one hacked device get root on other servers and add a rogue node.
Readers added context they thought people might want to know
The Tailscale SSH findings require prior authorized access on the tailnet per the researcher's report; Tailscale stated issues concern post-authorization behavior only and did not recommend disabling SSH for everyone.
tailworm.com
x.com/IntCyberDigest…
Very impressive!
🚀Excited to announce Nanbeige4.3-33B!
To push the performance ceiling of small agents, our strategy in Nanbeige4.3 starts with building a powerful teacher first.
Nanbeige4.3-33B delivers SOTA agentic performance among open models at ~30B parameters:
* Strong results on agentic coding and cowork benchmarks.
* When integrated into mainstream agent harnesses, it tackles real-world tasks with cost-efficient inference.
* It will serve as a teacher for upcoming smaller Nanbeige models, starting with Nanbeige4.3-3B.
huggingface.co/Nanbeige/Nanb…
huggingface.co/Nanbeige/Nanb…
#LLM #Agent
TheDavidTai retweeted
🔥 中午刚说 40 系还没支持，今晚 4090 到 4060Ti 全解锁了，连 FP8 都翻倍
40 系不能直接套 30 系的模块，是因为它换了另一套签名引导固件，多锁了四道权限门。今天研究者 TOTORONG 放出 AD10x 载荷，在驱动绑定前一次捅开四道门，而整个 40 系共用同一份引导固件，所以从 4090 到 4060 Ti 一下全通了——同样不刷 vBIOS、不改驱动。
1️⃣ RTX 4090：FP16/BF16 指令吞吐从 165.6 涨到 331.6 TFLOPS；500W 下跑 cuBLAS 8192³ 矩阵乘，TF32 从 77.5 涨到 165.9 TFLOPS（2.14 倍），3 张非公卡已复现
2️⃣ RTX 4080 SUPER：PyTorch 矩阵乘 FP16 从 110.6 涨到 202.7 TFLOPS（1.83 倍）；mmapeak 实测连 40 系独有的 FP8（FP32 累加）都从 201.8 涨到 376.1 TFLOPS
3️⃣ RTX 4070 Ti SUPER：240W 下 FP16/BF16 从 95 涨到 189.9 TFLOPS；跑 CUTLASS 4.8.1 原生 FP8 快累加矩阵乘从 158.5 涨到 309.1 TFLOPS（1.95 倍），322 万个输出单元零比特差异
4️⃣ RTX 4060 Ti 16GB 与 Win11：三卡同机 A/B 实测，182W 功耗墙下 BF16/FP16 仍涨 38–39%；Windows 11 也出了免核显、单卡直接亮机的 UEFI 引导版
💡 Linux 端装下方仓库（4080S 看原仓 PR #24）。40 系别用 targets=all，先跑 python3 tools/print_boot_option.py 拿到单卡参数加进内核启动项，关掉 Secure Boot，关机拔电源等几秒「冷启动」才生效；Windows 端去 Hugging Face 搜 TOTORONG/AD103_win_unlocker。
github.com/JustAResearcher/g…
中午还在等 40 系的各位，今晚敢拔电源试一把吗？
TheDavidTai retweeted
3090 Ti 和 A6000 是同一块芯片。
区别在卡里的一个标志位。
一个写着 FULL_SPEED，一个写着 HALF_SPEED。
Reddit 上有个哥们翻了很多代码，把这事讲明白了。
英伟达从 20 系到 50 系的游戏卡，FP16、BF16 做矩阵乘再用 FP32 累加，这条路径的速度都被砍了一半。
40 系和 50 系连 FP8 也砍了。
专业卡和数据中心卡都是全速。
被砍的刚好是 AI 最吃算力的那几块：
训练、微调 LoRA
文生图、文生视频这类扩散模型
大模型的 prefill，就是读长 prompt 那一步
一个人跟模型聊天、一个字一个字往外吐，基本没影响，那一步主要卡显存带宽。
这个哥们说，他的 RTX 6000 PRO 核心数只比 5090 多 10-15%，prefill 却快了 70% 左右。
GitHub 上已经有人把 30 系解锁了，800 多星。
做法是一个 Linux 内核模块，开机时抢在 NVIDIA 驱动接管之前，把速度选择器改掉。
3090 Ti 实测 BF16 从 87.5 TFLOPS 到 157.4，TF32 差不多翻倍。
40 系有人在 Windows 上跑通了 AD103，还得往别的型号移植。
50 系就难了，英伟达加了熔丝，专门读这个标志位。
有意思的是 50 系的 FP4 没被砍。
我觉得这事最值得琢磨的是定价。
同样的硅，同样的制造成本，价格差好几倍，差的是一个配置位。
特斯拉早年也这么干过，同一块电池用软件锁一截续航，加钱解锁。
卖硬件卖到最后，利润都在那个开关上。
原帖说解锁没风险，因为本来就是同一块 die。
我觉得保修和稳定性还是自己掂量一下。
看完突然觉得我那块吃灰的 3090 这几年一直在摸鱼。
而且是老板安排它摸的。
reddit.com/r/LocalLLaMA/comm…
TheDavidTai retweeted
Now I also confirmed that 2080ti unlock is successful with similar approach as the 30-series' method
it reach nearly same speed as 350w 3090(locked) with only 250w in mmapeak and mamf-finder
github.com/KohakuBlueleaf/Ko…
Confirmed that 3090 unlock is successful
with 360watt power wall, I got 75T -> 135T (measured in Pytorch)
Which is insane
I used github.com/Zagitta/gpu-unloc… with commit a9b0c75
Old mining GPUs with disabled ram and now 3090s with double the compute be like
Confirmed that 3090 unlock is successful
with 360watt power wall, I got 75T -> 135T (measured in Pytorch)
Which is insane
I used github.com/Zagitta/gpu-unloc… with commit a9b0c75
We look at LLM leak like we looked at new N64 games in Nintendo Power.
big Leak: Qwen4-27B architecture is being reverse-engineered from a merged kernel PR.
You might not need a MoE cluster for Qwen4-27B.
Intel’s vLLM XPU kernels just merged optimizations for the unreleased dense 27B: INT4 on 1-2 cards, MTP with 4 draft tokens, FP8 TP2, MTP, 4 residual streams/hyper-connections and QSA sparse attention kernels
The shapes are already leaking into the community.
Day-one local deploy energy is building.
