trying to manufacture inference hw + sw @fdotinc @scale_at_gmi
San Jose/Taipei
Joined December 2021
- Tweets1.9K
- Following7K
- Followers1.3K
- Likes36.1K
Pinned Tweet
I love inference! Tap in🧑🏻💻
Welcoming @solidwillity to SCALE! 🚀
His startup, Touchdown Labs, makes AI inference optimization simple and accessible for resource-constrained organizations.
Can't wait to see what you build!
#SCALEatGMI #GMICloud #Startup
William Chen retweeted
Frontier labs making custom chips is not really as hard / as impressive as you might think.
1. They are primarily targeting inference not training. inference is easier
2. They are targeting transformers only. Think large MXU matmul units
3. They are writing all kernels in close to ISA with coding agents RLed for their ISA.
4. They aren’t making high level compilers for codegen
A frontier lab vending a true generic chip like a Nvidia GPU or Google TPU is MUCH harder than spinning up an in house inference program.
Still I think you will see many more inference chip companies as the cost of the software is just the cost of tokens.
William Chen retweeted
Western team have independently verified that DeepSeek's performance numbers of 38$ dollar revenue per GPU at DeepSeek's rocket bottom prices is true.
This is a momemnt when you update your priors. At this rate with just 1 GW, you can make a revenue of 150B per year.
Even with lot of slack it would be 100B per year. If you use half the compute to come online by 2027 end, you will have 25GW for inference with potentially 2.5T revenue at DeepSeek rates and 10T at USA lab rates (world GDP is 125 T, 10T is 8% of it)
You answer for your-self are we overbuilding!!!
vLLM matches DeepInfer, DeepSeek's own engine for DeepSeek-V4.1-Flash, on DeepSeek's decode configuration across batch sizes 8 to 256, and does it at DEP32 against DeepInfer’s EP64DP64.
DeepInfer was originally built on vLLM. By integrating DeepSeek's open-source kernels and extensively optimizing prefill and decode, vLLM now matches DeepInfer's performance curves.
William Chen retweeted
Today I had in-depth conversations with optical, packaging, and system design engineers at Intel, NVIDIA, Marvell, Micron, Broadcom, and Ayar Labs.
My takeaway is that optics is just getting started. Don’t put a ceiling on how big it can get.
Investing is still hard. Conversations like these strengthen my conviction to keep going.
William Chen retweeted
If DeepSeek were American it would be on every magazine cover.
Think about it. A small lab with a fraction of the compute ships a frontier-class model under MIT. Then it open sources its production inference engine, its kernels, its file system. Publishes 155 trillion tokens a day and $38 of revenue per GPU-hour. Raises $15B anyway.
In America that is a national hero story. Senators would be posting about it. Every podcast would call it proof that open source made the country stronger.
Instead it got banned from US federal systems, called a security threat and accused of distillation, sometimes in the same week it gave away more of its production stack than OpenAI ever did.
The code is the same either way. The weights are the same. Only the flag changed the reaction.
I dont care about the flag. I care that I can download it, read it and run it on my own cards. By that measure DeepSeek did more for American developers this month than most American labs did all year.
William Chen retweeted
What the fuck are you even talking about? DeepSeek gave us a thousand times more than anyone else in this industry.
The weights, under MIT. Their production inference engine, under Apache 2.0, this week. Their attention kernels, their MoE communication library, their FP8 matmul library, their own distributed file system. The papers that explain how all of it works. And the component that makes your own deployment behave exactly like their API.
Now tell me what OpenAI and Anthropic gave us.
A $500 a month subscription. Limits that keep shrinking. Usage that drains faster than your plan says, until someone admits it was a bug. Models people swear got nerfed a month after launch. And a wave of account bans where nobody even tells you why.
Not 1 weight from Anthropic. Not 1 line of the code that serves ChatGPT.
Yes, the training data is not public, so call it open weights if you want. I do. But you picked the 1 lab that opened more of its real stack than anyone, and called it a lie.
Are you fucking out of your mind? You're the only one selling lies here.
DeepSeek sold you a lie.
Its new model, V4.1-Flash, is being sold as "open source".
It's not.
You can download it free, with no sign-up and use it in your business which sounds open source.
But you can't access the code they used to train it or the data they trained it on.
Without those, nobody can check how it was made or rebuild it and it’s over 700 billion parameters, so you can’t even run it on your laptop.
When you can build a business on something but can’t see how it was made, the right term is "open weights".
Call it what it is.
William Chen retweeted
Ben Horowitz explains why a16z doesn't invest in neoclouds after living through the dotcom bubble
"If you get into a pathological debt situation, that can wreck anybody. It's why, by the way, we're not invested in any of these neoclouds, for that reason. That whole thing scares me, having been through it."
"It can end up being a very dangerous arbitrage business where you can get upside down easily."
"You look at Loud Cloud just as an example, here's a simple thing that happened to us in 1999. Bandwidth was $1,500 a megabit. And you couldn't get it, it was very hard to get. And so you had to get a long-term contract, three years."
"By 2001 it was $50 a megabit. So if you were holding bandwidth, you were just dead."
"So that kind of thing is so out of the control of the entrepreneur. And so different than what we'd like to fund. That, you know, it just scared me to death."
William Chen retweeted
为什么我比较相信台积电对AI需求的判断？
因为这家公司经历过的半导体危机，实在太多了。
2001年互联网泡沫破裂，台积电第三季度产能利用率一度跌到41%。
2008年金融海啸，资本支出同比减少26%。
2023年半导体库存调整，资本支出又从363亿美元降到304亿美元。
但即使在行业低谷，台积电也没有停止先进制程的研发和升级。
经历了这么多轮周期，它应该比谁都清楚产能过剩的代价。
所以，当台积电计划2026年投入600–640亿美元，而且预计2026–2028年的资本支出还将显著高于过去三年时，我觉得这个信号值得认真研究。
不知道大家有没有注意到魏哲家10月15日法说会上的一段话？
台积电不只向客户了解需求，还会找客户的客户核实，甚至派自己的团队去美国，实地了解数据中心建设进度和电力供应。
云厂商的资本支出、融资能力，以及其他关键零组件的供应情况，也都是它评估需求的重要依据。
我觉得台积电还有一个经常被低估的能力，就是它多年积累的产业情报和需求分析体系。
从芯片设计、客户产品规划，到先进制程导入和未来量产安排，它掌握的产业信息，可能比大多数外部研究机构都要完整。
而且，这些信息还可以交叉验证。
以前做过多年供应链跟电子制造，我很清楚客户的Forecast和真正落地的订单是两回事。
尤其是半导体这种重资产行业，设备买回来、厂房建起来，客户就算推迟订单，折旧也不会跟着停下来。
所以，我不太相信台积电只是听客户讲了几个AI故事，就敢拿出这么多真金白银扩产。😂
如果一家经历过这么多轮半导体周期、拥有如此多一线产业信息的公司，经过反复验证后仍然决定大规模投资，我觉得这个判断很值得重视。
AI股票有没有泡沫？当然可能有。
但AI股票估值过高，和AI需求本身是不是泡沫，我觉得还是应该分开讨论。
比起市场每天冒出来的各种泡沫故事，我还是更愿意多看看产业链的实际变化。
这只是我的个人观察，也欢迎大家分享不同看法。👀
$TSM $NVDA
$TSM doesn’t build fabs on a whim. We’ve weathered semiconductor cycle after cycle, and TSMC is still standing.
On the ABF substrate side alone, Liwei is hearing that customers have already put down deposits to secure capacity through 2032.
Do you seriously think Taiwan’s suppliers would expand without customers putting money on the table first?
We in Taiwan have a stronger survival instinct than most. We know what’s at stake.
So stop fantasizing about the AI bubble bursting. As long as customers still have the cash and are willing to put it down to fight for capacity, this bubble isn’t bursting.
$INTC $MRVL $FORM $ASX $AEHR $AMAT
William Chen retweeted
菲尔兹奖得主、加州大学洛杉矶分校数学教授陶哲轩在最新访谈里罕见发火，直接点破了大模型刷榜背后一个没人收拾的烂摊子，
他说这场 AGI 竞赛正变得充满毒性，OpenAI 一口气倾倒出的 300 多篇论文，一夜之间端掉了年轻学者的初级研究课题。
他在访谈里算了一笔很无奈的账，面对这 300 多篇突如其来的 AI 论文，连他自己到现在也只顾得上看其中的 10 篇，
剩下的一大堆内容晦涩难懂，根本没人有精力仔细审完，甚至已经有一两篇因为被揪出错误而撤回。
外行看热闹的时候，习惯把 AI 批量证出数学定理当成技术突破来庆祝，觉得模型跑通了证明、发完新闻稿就算大功告成。
但陶哲轩直接戳破了这层泡沫，大厂为了多证几个定理、多发几篇公关稿、顺便测一把手里的新玩具，不惜把理解二字彻底抛在一边，这些 AI 只是在疯狂耗尽人类攒下的问题库，却没有给学科补回任何新的东西。
这就好比原本留给年轻学徒练手升级的新手村课题，被一台推土机连夜铲平了，
OpenAI 发完战报就觉得任务已经完成，也没有派人来协助清理，留给学术界的却是一堆没法拒绝的盲盒，里面好东西和垃圾混在一起，连比例是多少都没人摸得清，学者们反而得花海量时间去替 AI 验算排雷。
本质上，这给所有每天用 AI 写代码、做业务的人揭开了一个残酷的现实，
当生成内容的成本降到零，真正的瓶颈就瞬间变成了人类的审查带宽，不管是在顶尖科研还是日常工程开发里，只管倾倒未经验证的 AI 产出，不仅不是提效，反而是在给团队制造审核灾难。
不过看这段批评也得留个清醒，陶哲轩本人一直是数学界走在前列拥抱 AI 工具的学者，他一直在推动用 Lean 形式化语言辅助数学研究，
他真正反感的是大厂只管跑分公关、把人工核验成本甩锅给社区的做法，而不是否定 AI 参与科学探索本身。
当生成答案的成本变成零，不带理解的批量输出就不再是资产，而是全行业的审核负债。
知名博客主持人@dwarkesh_sp 最新的一期深度访谈，跟当世最顶尖的数学家陶哲轩，关于 AI 未来的深度对话，没有哗众取宠的爆点，没有套路化的叙事，只有顶尖大脑最真诚和最本质的思考，与其刷一下午的小视频，不如沉下心好好看完这场洗涤大脑和认知的对话
William Chen retweeted
Chamath says Anthropic should sacrifice valuation to raise at least $200 billion, because even that barely buys the compute needed to survive a correction
“Here’s my very specific thing about why I think they should price it to clear the market for $200 billion—and I’m specific about that number. They need, in my estimate, at least $200 billion. If you want to be super safe, a quarter-trillion gets the job done.
“What does the price need to be to attract that incremental dollar off the sidelines? Can they raise $80 billion in an IPO at $1.7, $1.8 or $1.9 trillion? I think so, handily. Can they raise $100 billion? Maybe it’s pushing it. At $150 billion, I think you need to be back down to $1.5 trillion.”
JCal: “You’re saying you want them to raise more money at a lower valuation so they have a war chest.”
Chamath: “$150 billion at $1.5 trillion is not enough. $200 billion barely gets them a few gigawatts. That is the bottom-line math if Anthropic is going to go through and survive an eventual correction.
“I think it’s much better for them to have $200 billion to $250 billion today than to try to ring the bell at $1.9 or $2 trillion for $80 billion, file a mixed shelf—which takes about six months—and hope the markets are in line.
“I just think there’s more risk in the future than there is today. Why take the risk? Price it to clear the market.”
Ben Horowitz reveals Amazon CEO Andy Jassy told him its AI compute is sold out beyond 2028, and says that shortage makes this fundamentally different from the dot-com bubble
“This shortage is so bad. You know, I was speaking with Andy Jassy, who runs Amazon, yesterday. And I said, yeah, like, we’re out of everything, you know, till 2028. He’s like, oh, it’s beyond 2028. Like, I don’t think we get stuff back, like, end of 2028, maybe. So everything else is sold out.”
“So he has the advantage of that, he’s got supply in a great shortage, which is extremely different than the internet bubble. The internet bubble, we didn’t have enough demand, because we had to build the internet, people had to get broadband, you know, you needed all this stuff people had to buy, you know, originally, they had to get their browser at price, all this kind of thing.”
“There weren’t enough consumers on the internet to absorb all the bandwidth that had gotten built out. And they had a massive amount of debt, not unlike we’re building today, although not as big. But the lack of demand killed it. We don’t really have a lack of demand in AI. So I think the dynamics will be different.”
“I do think, like, for sure, if you get into a pathological debt situation, that can wreck anybody. It’s why, by the way, we’re not invested in any of these neo-clouds either, for that reason.”
It’s up to me
Reindustrialization isn’t buying a $320 made in the USA sweater that can be bought for less than $10 in multiple other countries.
It’s not about getting a billion dollar valuation on your startup that has never built a real product that has a multi-million dollar contract from a government that spent 2 trillion more than it took in this year.
It’s not about looking cool** and it’s not about getting asset-price rich on mystery money.
Reindustrialization is about 4 things.
Until we have those 4, it ain’t going to happen.
Let me tell you a short story.
We had a supplier in China with a coal boiler.
The coal boiler belched dark soot into the air.
One day the government came in and said “you can’t use your coal boiler anymore. Replace it with a natural gas one; the coal is dirtying up the air.”
The factory would have to shut down for months.
No money for the boss.
No work for the employees.
And no orders for me.
“How long will the orders be delayed?”
“They won’t.”
“And how will you do that?”
“Don’t worry about it.”
The orders shipped on time.
How?
The factory switched to working nights.
The black smoke from the coal boiler couldn’t be seen in the dark night sky.
At the same time they installed a new natural gas boiler during the day.
By the way, this boiler was bigger than two semi trailer trucks side by side, literally it is beyond most people’s comprehension.
And that’s the point.
Reindustrialization is beyond peoples’ comprehension in America.
1. You need to have a government that does the right thing
2. You need to have factory ownership that is hungry for business
3. You need to have workers who are willing to make big sacrifices
4. And then everyone around them needs to be able tolerate the black smoke from the coal boiler
We don’t have any of these things today.
Will that change?
Maybe, but overpriced sweaters and billion dollar valuations on vaporware is not how we’re going to get there.
William Chen retweeted
Old friends are the best friends. Jensen rocks.
The Star Mind computer being co-engineered by SpaceX and Nvidia is far better than almost anyone expects and will reach volume production next year.
Working with the incredibly talented Nvidia engineering team is pure joy!
Jensen Huang says he loves working with Elon Musk, and reveals their engineering teams are designing a new type of server to put data centers in space
“I’m delighted that I love working with Elon. He’s using our technology for his own AI in training models and such, deploying models, of course."
"Grok Bot is super cool. The demand for Grok Bot, the consumption for Grok Bot is really skyrocketing. And then, of course, he rents it."
"And so the combination of these three things, he’s fairly unique in doing that. I guess Google does that. That would be yet another example."
"And so I think he also has the advantage that he loves the intersection between computing and the physical buildouts. He loves that whole area. Nobody knows more about energy than he does."
"One particular new market that we’re going to go explore together is putting data centers out in space. Our two engineering teams are working super closely together to create a new type of server, a new type of architecture that might be super good in space and on Earth.”
William Chen retweeted
Tim Cook's take on why competitors can't copy Apple is sharper than most people realize.
They start by copying the product. Then they realize the product isn't the hard part.
Cook said it directly: people who try to copy Apple first try to copy what they can see. The hardware. The design. The packaging. And they can get close on all of it.
But the thing that made the product — the organizational culture, the decision-making, the obsessive prioritization — that's invisible from the outside. You can't find it in a teardown.
This is why Apple's moat has held for decades while better-funded competitors kept losing. Samsung had more SKUs. Google had more engineers. Amazon had more distribution. None of it translated into the same output because none of it replicated the same input.
The lesson for founders is uncomfortable: if your competitive advantage is visible in your product, it's already copyable. The real moat is the culture that makes the product — and that takes decades to build, not quarters.
Reverse-engineering the iPhone is an engineering problem. Reverse-engineering how Apple decides what to build is a different problem entirely.
Bc Gen Z American kids wanted to drop ship and do ecom
But don’t worry I’m making math and science cool in America
Missing piece to reindustrialize America is the culture to make STEM and hard work cool for the younger generation to aspire to
And then building and maintaining the ecosystem around it
“When I’m walking around in the AI labs in San Francisco, maybe like a third, maybe even half the folks have Chinese heritage. Many of them are Chinese nationals...” says @danwwang, research fellow at the Hoover Institution. “There’s something that China and Russia [have] that fewer American students are interested in,” Wang continues, “and that is math.”
Most accurate take
China’s $CXMT just sent Korean memory stocks tumbling, and the market decided to dump the entire semiconductor sector.
That makes no sense to me.
If China eventually floods the memory market with cheaper supply, memory pricing power gets crushed. But cheaper memory doesn’t mean less AI infrastructure spending.
Relatively, Lower memory costs improve data center economics and free up capital for other components: compute, networking, optics, power, cooling, and testing.
The value doesn’t disappear. It shifts across the supply chain.
The market is treating a potential margin headwind for memory makers as a demand shock for the entire semiconductor industry.
Those are two very different things.
One company’s pricing pressure is another company’s opportunity.
I’m watching where the money flows next. $FORM $TER $AEHR
Good night 💤 🌙
Playing it safe in the semis make sense bc you don't want to lose billions and years of resources
But there's different approaches to derisking it so it doesn't fall on one company or even handful of companies
If you want to push the frontier, It's time to take more risks across the board
CXMT has been forced to pull off an engineering miracle and if it pulls it off then it might encourage more risk taking
BofA on CXMT's 4F2 architecture:
Restricted from importing ASML’s EUV lithography tools due to US sanctions, CXMT is likely advancing toward 4F2 architecture faster than the "Big 3" DRAM manufacturers.
While the top three producers have already shifted to EUV-based 6F2 designs at low- to mid-10nm nodes and have researched 4F2 for over a decade, they have avoided mass production.
This hesitation stems from poor manufacturing yields, steep costs, and technical challenges—such as extreme heat and structural fragility when cramming billions of transistors and capacitors into a compact die.
$MU $DRAM $EWY
Don't worry everyone I'm solving this
CPO Testing Faces A Moving Target Problem
semiengineering.com/cpo-test…
Test vendors adapt to changing connector architectures & extreme alignment tolerances as CPO heads toward mainstream
#CPO #CPOtest #NPO @Teradyneinc #Advantest
@aseglobal @Keysight #semiconductor #datacenter
