trying to manufacture inference hw + sw @fdotinc @scale_at_gmi
San Jose/Taipei
Joined December 2021
- Tweets1.9K
- Following7K
- Followers1.3K
- Likes36.1K
William Chen retweeted
Frontier labs making custom chips is not really as hard / as impressive as you might think.
1. They are primarily targeting inference not training. inference is easier
2. They are targeting transformers only. Think large MXU matmul units
3. They are writing all kernels in close to ISA with coding agents RLed for their ISA.
4. They aren’t making high level compilers for codegen
A frontier lab vending a true generic chip like a Nvidia GPU or Google TPU is MUCH harder than spinning up an in house inference program.
Still I think you will see many more inference chip companies as the cost of the software is just the cost of tokens.
William Chen retweeted
Western team have independently verified that DeepSeek's performance numbers of 38$ dollar revenue per GPU at DeepSeek's rocket bottom prices is true.
This is a momemnt when you update your priors. At this rate with just 1 GW, you can make a revenue of 150B per year.
Even with lot of slack it would be 100B per year. If you use half the compute to come online by 2027 end, you will have 25GW for inference with potentially 2.5T revenue at DeepSeek rates and 10T at USA lab rates (world GDP is 125 T, 10T is 8% of it)
You answer for your-self are we overbuilding!!!
vLLM matches DeepInfer, DeepSeek's own engine for DeepSeek-V4.1-Flash, on DeepSeek's decode configuration across batch sizes 8 to 256, and does it at DEP32 against DeepInfer’s EP64DP64.
DeepInfer was originally built on vLLM. By integrating DeepSeek's open-source kernels and extensively optimizing prefill and decode, vLLM now matches DeepInfer's performance curves.
William Chen retweeted
Today I had in-depth conversations with optical, packaging, and system design engineers at Intel, NVIDIA, Marvell, Micron, Broadcom, and Ayar Labs.
My takeaway is that optics is just getting started. Don’t put a ceiling on how big it can get.
Investing is still hard. Conversations like these strengthen my conviction to keep going.
William Chen retweeted
If DeepSeek were American it would be on every magazine cover.
Think about it. A small lab with a fraction of the compute ships a frontier-class model under MIT. Then it open sources its production inference engine, its kernels, its file system. Publishes 155 trillion tokens a day and $38 of revenue per GPU-hour. Raises $15B anyway.
In America that is a national hero story. Senators would be posting about it. Every podcast would call it proof that open source made the country stronger.
Instead it got banned from US federal systems, called a security threat and accused of distillation, sometimes in the same week it gave away more of its production stack than OpenAI ever did.
The code is the same either way. The weights are the same. Only the flag changed the reaction.
I dont care about the flag. I care that I can download it, read it and run it on my own cards. By that measure DeepSeek did more for American developers this month than most American labs did all year.
William Chen retweeted
What the fuck are you even talking about? DeepSeek gave us a thousand times more than anyone else in this industry.
The weights, under MIT. Their production inference engine, under Apache 2.0, this week. Their attention kernels, their MoE communication library, their FP8 matmul library, their own distributed file system. The papers that explain how all of it works. And the component that makes your own deployment behave exactly like their API.
Now tell me what OpenAI and Anthropic gave us.
A $500 a month subscription. Limits that keep shrinking. Usage that drains faster than your plan says, until someone admits it was a bug. Models people swear got nerfed a month after launch. And a wave of account bans where nobody even tells you why.
Not 1 weight from Anthropic. Not 1 line of the code that serves ChatGPT.
Yes, the training data is not public, so call it open weights if you want. I do. But you picked the 1 lab that opened more of its real stack than anyone, and called it a lie.
Are you fucking out of your mind? You're the only one selling lies here.
DeepSeek sold you a lie.
Its new model, V4.1-Flash, is being sold as "open source".
It's not.
You can download it free, with no sign-up and use it in your business which sounds open source.
But you can't access the code they used to train it or the data they trained it on.
Without those, nobody can check how it was made or rebuild it and it’s over 700 billion parameters, so you can’t even run it on your laptop.
When you can build a business on something but can’t see how it was made, the right term is "open weights".
Call it what it is.
William Chen retweeted
Ben Horowitz explains why a16z doesn't invest in neoclouds after living through the dotcom bubble
"If you get into a pathological debt situation, that can wreck anybody. It's why, by the way, we're not invested in any of these neoclouds, for that reason. That whole thing scares me, having been through it."
"It can end up being a very dangerous arbitrage business where you can get upside down easily."
"You look at Loud Cloud just as an example, here's a simple thing that happened to us in 1999. Bandwidth was $1,500 a megabit. And you couldn't get it, it was very hard to get. And so you had to get a long-term contract, three years."
"By 2001 it was $50 a megabit. So if you were holding bandwidth, you were just dead."
"So that kind of thing is so out of the control of the entrepreneur. And so different than what we'd like to fund. That, you know, it just scared me to death."
William Chen retweeted
为什么我比较相信台积电对AI需求的判断？
因为这家公司经历过的半导体危机，实在太多了。
2001年互联网泡沫破裂，台积电第三季度产能利用率一度跌到41%。
2008年金融海啸，资本支出同比减少26%。
2023年半导体库存调整，资本支出又从363亿美元降到304亿美元。
但即使在行业低谷，台积电也没有停止先进制程的研发和升级。
经历了这么多轮周期，它应该比谁都清楚产能过剩的代价。
所以，当台积电计划2026年投入600–640亿美元，而且预计2026–2028年的资本支出还将显著高于过去三年时，我觉得这个信号值得认真研究。
不知道大家有没有注意到魏哲家10月15日法说会上的一段话？
台积电不只向客户了解需求，还会找客户的客户核实，甚至派自己的团队去美国，实地了解数据中心建设进度和电力供应。
云厂商的资本支出、融资能力，以及其他关键零组件的供应情况，也都是它评估需求的重要依据。
我觉得台积电还有一个经常被低估的能力，就是它多年积累的产业情报和需求分析体系。
从芯片设计、客户产品规划，到先进制程导入和未来量产安排，它掌握的产业信息，可能比大多数外部研究机构都要完整。
而且，这些信息还可以交叉验证。
以前做过多年供应链跟电子制造，我很清楚客户的Forecast和真正落地的订单是两回事。
尤其是半导体这种重资产行业，设备买回来、厂房建起来，客户就算推迟订单，折旧也不会跟着停下来。
所以，我不太相信台积电只是听客户讲了几个AI故事，就敢拿出这么多真金白银扩产。😂
如果一家经历过这么多轮半导体周期、拥有如此多一线产业信息的公司，经过反复验证后仍然决定大规模投资，我觉得这个判断很值得重视。
AI股票有没有泡沫？当然可能有。
但AI股票估值过高，和AI需求本身是不是泡沫，我觉得还是应该分开讨论。
比起市场每天冒出来的各种泡沫故事，我还是更愿意多看看产业链的实际变化。
这只是我的个人观察，也欢迎大家分享不同看法。👀
$TSM $NVDA
$TSM doesn’t build fabs on a whim. We’ve weathered semiconductor cycle after cycle, and TSMC is still standing.
On the ABF substrate side alone, Liwei is hearing that customers have already put down deposits to secure capacity through 2032.
Do you seriously think Taiwan’s suppliers would expand without customers putting money on the table first?
We in Taiwan have a stronger survival instinct than most. We know what’s at stake.
So stop fantasizing about the AI bubble bursting. As long as customers still have the cash and are willing to put it down to fight for capacity, this bubble isn’t bursting.
$INTC $MRVL $FORM $ASX $AEHR $AMAT
William Chen retweeted
菲尔兹奖得主、加州大学洛杉矶分校数学教授陶哲轩在最新访谈里罕见发火，直接点破了大模型刷榜背后一个没人收拾的烂摊子，
他说这场 AGI 竞赛正变得充满毒性，OpenAI 一口气倾倒出的 300 多篇论文，一夜之间端掉了年轻学者的初级研究课题。
他在访谈里算了一笔很无奈的账，面对这 300 多篇突如其来的 AI 论文，连他自己到现在也只顾得上看其中的 10 篇，
剩下的一大堆内容晦涩难懂，根本没人有精力仔细审完，甚至已经有一两篇因为被揪出错误而撤回。
外行看热闹的时候，习惯把 AI 批量证出数学定理当成技术突破来庆祝，觉得模型跑通了证明、发完新闻稿就算大功告成。
但陶哲轩直接戳破了这层泡沫，大厂为了多证几个定理、多发几篇公关稿、顺便测一把手里的新玩具，不惜把理解二字彻底抛在一边，这些 AI 只是在疯狂耗尽人类攒下的问题库，却没有给学科补回任何新的东西。
这就好比原本留给年轻学徒练手升级的新手村课题，被一台推土机连夜铲平了，
OpenAI 发完战报就觉得任务已经完成，也没有派人来协助清理，留给学术界的却是一堆没法拒绝的盲盒，里面好东西和垃圾混在一起，连比例是多少都没人摸得清，学者们反而得花海量时间去替 AI 验算排雷。
本质上，这给所有每天用 AI 写代码、做业务的人揭开了一个残酷的现实，
当生成内容的成本降到零，真正的瓶颈就瞬间变成了人类的审查带宽，不管是在顶尖科研还是日常工程开发里，只管倾倒未经验证的 AI 产出，不仅不是提效，反而是在给团队制造审核灾难。
不过看这段批评也得留个清醒，陶哲轩本人一直是数学界走在前列拥抱 AI 工具的学者，他一直在推动用 Lean 形式化语言辅助数学研究，
他真正反感的是大厂只管跑分公关、把人工核验成本甩锅给社区的做法，而不是否定 AI 参与科学探索本身。
当生成答案的成本变成零，不带理解的批量输出就不再是资产，而是全行业的审核负债。
知名博客主持人@dwarkesh_sp 最新的一期深度访谈，跟当世最顶尖的数学家陶哲轩，关于 AI 未来的深度对话，没有哗众取宠的爆点，没有套路化的叙事，只有顶尖大脑最真诚和最本质的思考，与其刷一下午的小视频，不如沉下心好好看完这场洗涤大脑和认知的对话
William Chen retweeted
Chamath says Anthropic should sacrifice valuation to raise at least $200 billion, because even that barely buys the compute needed to survive a correction
“Here’s my very specific thing about why I think they should price it to clear the market for $200 billion—and I’m specific about that number. They need, in my estimate, at least $200 billion. If you want to be super safe, a quarter-trillion gets the job done.
“What does the price need to be to attract that incremental dollar off the sidelines? Can they raise $80 billion in an IPO at $1.7, $1.8 or $1.9 trillion? I think so, handily. Can they raise $100 billion? Maybe it’s pushing it. At $150 billion, I think you need to be back down to $1.5 trillion.”
JCal: “You’re saying you want them to raise more money at a lower valuation so they have a war chest.”
Chamath: “$150 billion at $1.5 trillion is not enough. $200 billion barely gets them a few gigawatts. That is the bottom-line math if Anthropic is going to go through and survive an eventual correction.
“I think it’s much better for them to have $200 billion to $250 billion today than to try to ring the bell at $1.9 or $2 trillion for $80 billion, file a mixed shelf—which takes about six months—and hope the markets are in line.
“I just think there’s more risk in the future than there is today. Why take the risk? Price it to clear the market.”
Ben Horowitz reveals Amazon CEO Andy Jassy told him its AI compute is sold out beyond 2028, and says that shortage makes this fundamentally different from the dot-com bubble
“This shortage is so bad. You know, I was speaking with Andy Jassy, who runs Amazon, yesterday. And I said, yeah, like, we’re out of everything, you know, till 2028. He’s like, oh, it’s beyond 2028. Like, I don’t think we get stuff back, like, end of 2028, maybe. So everything else is sold out.”
“So he has the advantage of that, he’s got supply in a great shortage, which is extremely different than the internet bubble. The internet bubble, we didn’t have enough demand, because we had to build the internet, people had to get broadband, you know, you needed all this stuff people had to buy, you know, originally, they had to get their browser at price, all this kind of thing.”
“There weren’t enough consumers on the internet to absorb all the bandwidth that had gotten built out. And they had a massive amount of debt, not unlike we’re building today, although not as big. But the lack of demand killed it. We don’t really have a lack of demand in AI. So I think the dynamics will be different.”
“I do think, like, for sure, if you get into a pathological debt situation, that can wreck anybody. It’s why, by the way, we’re not invested in any of these neo-clouds either, for that reason.”
William Chen retweeted
Old friends are the best friends. Jensen rocks.
The Star Mind computer being co-engineered by SpaceX and Nvidia is far better than almost anyone expects and will reach volume production next year.
Working with the incredibly talented Nvidia engineering team is pure joy!
Jensen Huang says he loves working with Elon Musk, and reveals their engineering teams are designing a new type of server to put data centers in space
“I’m delighted that I love working with Elon. He’s using our technology for his own AI in training models and such, deploying models, of course."
"Grok Bot is super cool. The demand for Grok Bot, the consumption for Grok Bot is really skyrocketing. And then, of course, he rents it."
"And so the combination of these three things, he’s fairly unique in doing that. I guess Google does that. That would be yet another example."
"And so I think he also has the advantage that he loves the intersection between computing and the physical buildouts. He loves that whole area. Nobody knows more about energy than he does."
"One particular new market that we’re going to go explore together is putting data centers out in space. Our two engineering teams are working super closely together to create a new type of server, a new type of architecture that might be super good in space and on Earth.”
William Chen retweeted
Tim Cook's take on why competitors can't copy Apple is sharper than most people realize.
They start by copying the product. Then they realize the product isn't the hard part.
Cook said it directly: people who try to copy Apple first try to copy what they can see. The hardware. The design. The packaging. And they can get close on all of it.
But the thing that made the product — the organizational culture, the decision-making, the obsessive prioritization — that's invisible from the outside. You can't find it in a teardown.
This is why Apple's moat has held for decades while better-funded competitors kept losing. Samsung had more SKUs. Google had more engineers. Amazon had more distribution. None of it translated into the same output because none of it replicated the same input.
The lesson for founders is uncomfortable: if your competitive advantage is visible in your product, it's already copyable. The real moat is the culture that makes the product — and that takes decades to build, not quarters.
Reverse-engineering the iPhone is an engineering problem. Reverse-engineering how Apple decides what to build is a different problem entirely.
William Chen retweeted
Travis Kalanick on the founder's paradox: the better you get, the harder it is to stay fired up.
"The young entrepreneurs that are coming up right now don't totally get it."
"When you get into your 3rd and 4th company, I don't know, this may be even my 5th at this point, you get fricking really good at doing stuff."
"Things that used to take me a day or three days, a lot of stress, anxiety, like, 'How am I gonna do this?' take me like 45 minutes. And that's a beautiful thing."
"I'm so used to adversity at this point that it's kind of normal, so I don't get as pissed off. But being pissed off makes you good as an entrepreneur, so I gotta make sure I'm still fired up."
"You get too used to adversity, and that's kind of a problem."
Full conversation: youtube.com/watch?v=r8qKNFeB…
@travisk
William Chen retweeted
There are really only two ways to make mid-volume manufacturing economical:
1- The China method: Cheap labor, ruthlessly efficient operations. Works great in China. Good luck replicating it with US labor costs.
2- The technology method: Reinvent manufacturing so the technology itself can produce hundreds to thousands of complex parts cost-effectively. Good news is, this is America’s strength.
Everything else is mostly “PowerPoint engineering.”
Of course, PowerPoint engineering has learned to dress itself up in more respectable language: financial engineering, patriotism, and the occasional massive equity round backing a team with a compelling “can-do” attitude.
America has a manufacturing problem. Correct diagnosis.
Prototyping is the solution? Wrong. That was never the hard part.
Mass production and capacity? Sounds smarter, but high-volume manufacturing is a solved playbook once demand justifies the investment.
The real problem everyone keeps dancing around is the missing middle: making hundreds to thousands of complex parts fast, affordably, and without spending a fortune on bad unit economics.
Prototypes make headlines. Mass production makes incumbents richer. The missing middle is where the next generation of American industry gets built.
William Chen retweeted
DOJ says this was a Saudi National
We discovered a sophisticated recording device that was hidden inside one of our offices. We used our models to understand the device and where it had come from, identifying that a specific employee had planted it a few days earlier. We referred the matter to the FBI, and we’re grateful to the agents and prosecutors who moved quickly to make an arrest. The FBI’s investigation is ongoing and we will continue to support their work.
At this time, we have no indications of this individual inappropriately accessing or exfiltrating model weights, source code, or customer data. The employee was terminated.
We take protecting our technology, data, and people seriously, and as a matter of national importance. Our security program is designed to protect against a wide range of threat actors, from insider risks to cybercriminals to nation states, and this case is an example of how AI can help address these sophisticated security threats.
William Chen retweeted
America has a manufacturing problem. Correct diagnosis.
Prototyping is the solution? Wrong. That was never the hard part.
Mass production and capacity? Sounds smarter, but high-volume manufacturing is a solved playbook once demand justifies the investment.
The real problem everyone keeps dancing around is the missing middle: making hundreds to thousands of complex parts fast, affordably, and without spending a fortune on bad unit economics.
Prototypes make headlines. Mass production makes incumbents richer. The missing middle is where the next generation of American industry gets built.
William Chen retweeted
Palmer Luckey explains why China is advancing so rapidly in technology and why the United States is struggling to keep up 🇺🇸🇨🇳
“The reason China is doing so well with advanced technology isn’t that they have a lot of people. It’s that they made a series of decisions that make high-tech companies much faster and much more effective.
They have very low-cost energy. You can complain about all of their coal plants, but the net output is that they have a very, very low cost of energy.
You have a really favorable labor environment. You have a very favorable logistics and supply chain world. You have very favorable laws regarding the extraction and processing of metals, minerals, and rare earths.
People ask me, ‘Palmer, what can we do to catch up with China?’
I say, ‘We need way more energy. We’re probably going to need to burn a lot more fossil fuels, and we’re probably going to need to relax labor laws.’
And they say, ‘I want to catch up with China, but I don’t want to do any of the things they’re doing to lead.’
You have to make a choice at some point. It’s not really possible to say, ‘I want to win, but I’m not willing to do anything to do that.’”
William Chen retweeted
BREAKING: California is building its first new steel mill in over half a century.
Pacific Steel Group is investing $700 million in a steel production facility in Kern County, designed to melt recycled steel and manufacture rebar for critical infrastructure projects.
ALT Source: State of California
William Chen retweeted
Exactly.
The key lesson is that when China becomes a serious player in an industry, the result is often a dramatic compression of margins across the board. We’ve seen this play out in LEDs, EVs, and lithium-ion batteries. Memory could be next, followed by AI models and, eventually, semiconductor manufacturing itself.
When China figures out how to manufacture a product, their ability to scale is often greater than the rest of the world combined. We saw this with lithium batteries and solar, for example.
SpaceX is a rare counter-example, where its mass to orbit is roughly an order of magnitude greater than the rest of the world, BUT China now does a majority of non-SpaceX mass to orbit. SpaceX is an anomaly in the matrix.
Maybe RAM will be different, but RAM was a low margin commodity only ~3 years ago. LP5 and LP6 are designed to a common standard that China seems to be solving fast.
Newer HBM architectures and things like an SRAM layer below logic and 3D stacking RAM on top of logic while finding a way to keep the logic cooled will be harder to solve.
William Chen retweeted
It is increasingly obvious that we are at the dawn of a silicon cambrian explosion where enormous numbers of novel architectures and ASICs are going to be explored. The idea of compute as a commodity will quickly go away as generalization is traded for use-specific silicon (as we are beginning to see with differentiated training vs inference chips). Silicon optimized for power, optimized for latency, optimized for security, optimized for radiation, etc.
Those best positioned are at the fab layer. Packaging, interconnects, networking, etc. are going to become extremely valuable and require enormous scaling. Elons bet on Terafab was a remarkably timely move.
Today we are announcing our biggest bet yet on our own silicon.
Starting next calendar year, we will scale our Maia family to multi-gigawatt deployments, making it the primary inference system for our own product and services.
And we’re introducing Maia 300, which delivers 40% more tokens per watt compared to leading GPU-based systems.
We are already deploying Maia 200 across the US, Europe, and Australia, delivering up to 1.7x more tokens per watt across experiences in Teams and Copilot.
Our goal is to build on the full-stack efficiency gains we’ve already seen, while continuing to bring the latest innovations from across the ecosystem into our fleet.
William Chen retweeted
It's becoming clear now. Margins will be managed by vertical integration of silicon. Everyone is doing it.
Congrats Microsoft.
Today we are announcing our biggest bet yet on our own silicon.
Starting next calendar year, we will scale our Maia family to multi-gigawatt deployments, making it the primary inference system for our own product and services.
And we’re introducing Maia 300, which delivers 40% more tokens per watt compared to leading GPU-based systems.
We are already deploying Maia 200 across the US, Europe, and Australia, delivering up to 1.7x more tokens per watt across experiences in Teams and Copilot.
Our goal is to build on the full-stack efficiency gains we’ve already seen, while continuing to bring the latest innovations from across the ecosystem into our fleet.