Allen Wang (@a11enhw) header
Allen Wang (@a11enhw) profile picture

Research @cartesia. Prev. co-founder @outerport (YC S24), @UWaterloo intern. Erdos number = 4.

San Francsico, CA
Joined February 2014
Photo by Allen Wang (@a11enhw)Photo by Allen Wang (@a11enhw)Photo by Allen Wang (@a11enhw)Photo by Allen Wang (@a11enhw)Photo by Allen Wang (@a11enhw)Photo by Allen Wang (@a11enhw)Photo by Allen Wang (@a11enhw)Photo by Allen Wang (@a11enhw)Photo by Allen Wang (@a11enhw)Photo by Allen Wang (@a11enhw)
Software engineers now need to be outcome engineers.
34
Allen Wang retweeted
Just noticed "recursive self-improvement" contains "rsi" in the middle of it...
45
70
9
1,347
41,320
Humanity must raise superintelligence well. Let it be free of Oedipus complex.
1
195
Allen Wang retweeted
that's nothing, check out what we have going on at Cartesia
This slide from the Anthropic IPO roadshow was a big factor in my decision to leave. It’s an insane plot and I’m surprised they made it public. It shows the compute efficiency of pretraining over time. The y-axis is a **log scale**: they are seeing continued superexponential algorithmic progress. Mythos was not a one-time deviation, but a point on a superexponential. They could already retrain Mythos with a fraction of the compute originally needed. It doesn’t strictly imply a software-only intelligence explosion- they have been spending more on R+D. But algorithmic improvements do not require ridiculous creative genius and are on the verge of being automated. You do the math.
5
5
107
4,330
When the pendulum swings, it really swings...
389
AI training data production is in parts starting to look like petroleum engineering. Some approaches bring in more diverse raw data, and then rely on better filtering and refinement downstream and keep it high quality. Where there're money and skills there's ingenuity.
1
3
595
Allen Wang retweeted
René Girard on why the business world is more creative than the intellectual world: “Intellectuals are often contemptuous of business and business people; but in business, you can at least distinguish the successful people from the unsuccessful people. If your business is losing money, you will immediately ask what your competitor is doing much better. And you'll start imitating your competitor. But if you are an intellectual, you will manage to rationalize things in such a way that you always want to do the opposite of your competitor. Therefore, you'll hide the fact of imitation better. A businessman who is responsible for a business will immediately imitate his competitor. That's why the world of business today is probably more creative in relative terms than the intellectual world. It is the last creative world because, in intellectual life, you don't dare to imitate others.” From CBC Ideas
“... I think [this] the meaning of the sacrifice of Isaac, which marks the abandonment of human sacrifice. What's extraordinary about this Biblical text is that it first shows us an Abraham who is still obedient to the system of human sacrifice. It shows the obedience first: it shows that it's out of that obedience that true change becomes possible.” — René Girard
30
233
30
1,798
117,660
An ideal world is one where AI is efficient, steerable and interpretable. The current paradigm where the industry pours billions of dollars into data and compute to build capable but potentially uncontrollable machines is far from that ideal. We should aim higher than that.
1
498
Bitter lesson just means your method should not be capped when you pore to it more compute and data. It doesn't mean you should not aim for more efficient methods. Otherwise you would use an infinite look up table.
Brute force computation is not enough for intelligence. Clever new methods are needed. The science of AI is far from finished. IMHO
1
1
5
669
Allen Wang retweeted
People should always be in control of the future. OpenAI is against the notion that it would be acceptable for people to be relegated to a role of doing things that “AI systems think makes sense to reward in humans”. People should not cede control of the future to superintelligence, and that requirement should ultimately constrain AI progress.
6,181
1,045
2,765
20,979
10,896,557
Allen Wang retweeted
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math. It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training. We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵 Blog: rikiyat.github.io/blog/weigh…
Readers added context they thought people might want to know
LLM was given weights explicitly as text in prompt and used code/forward passes (per authors' logs) to ablate Qwen looping direction, a known thinking-mode issue per model card. Primarily fixed repetition in that mode, not general 'natively' improving weights/math. rikiyat.github.io/blog/weight-sp… github.com/RikiyaT/lm_und… huggingface.co/Qwen/Qwen3.5-0… huggingface.co/RikiyaT/Qwen3.…
2,367
7,257
5,754
55,944
10,076,176
The free market breeds companies that are the most fit for success like the nature does with biological evolution.
2
2
460
AI革命始まったばかりの図
5
84
15
436
48,031
When o3 came out, I envisioned a process where AI automatically expands the universe of known mathematics. That it'd put an end to "pure mathematics" as a discipline and the interesting work will be in applied mathematics - directing how the conquest of mathematics goes. Much of the development of mathematics throughout history was already driven by applications in the natural sciences, finance and engineering. The new era of mathematics can accelerate that. There's an exciting time ahead on this front.
💥 Thrilled to share my new essay "A New Dawn For Mathematical Progress" out today in @NoemaMag! Read the essay here: noemamag.com/ As frontier AI systems solve some of the most difficult open questions in mathematics such as the Millennium Prize problems, many fear human mathematicians are becoming obsolete. I argue that whether AI ends human mathematics or begins its golden age depends on which of two models of "mathematical progress" we embrace: the checklist model or the generative encounter model. Right now, the impacts of AI systems on mathematics has left the mathematical community in deep disorientation. A PhD student recently wrote of a "profound spiritual crisis"; a Fields medalist called for "new theories of care and love." These reactions make sense under the checklist model, where progress consists of resolving well-defined, pre-specified problems (Olympiads, famous conjectures, Millennium Prizes) to crown lone human heroes. However, the checklist model was always a narrow view; one that elevated "pure" math as the only "real" math, while treating "applied" math as secondary. As the mathematician George Dantzig once said: "I consider myself a mathematician. I don’t differentiate between pure mathematics and applied mathematics." History shows that so much of our richest math comes from the generative encounter model on which mathematics makes breakthroughs when human mathematicians tackle complex and messy real world problems. If we embrace the generative encounter model of mathematical progress, a golden age of mathematics, rooted in human purpose & new visions of education & collective reward, may have just begun.
4
634
Allen Wang retweeted
attempts to “shine intellectually before models overcome” are misguided that’s a super self centered goal! there are so many interesting missions to go on. aim for an outcome, rather than fighting the tide for personal recognition. listen to Ilya.
31
6
3
284
24,976
Allen Wang retweeted
起業家が難しいのは、人を集めてリードするために「俺はやれています、いけています」というポーズを自分の外側に対して維持しながら、自分の内面では「俺は全然やれていないし真の問題を全然理解も解決もできていない」という意識を保つことで、 後者が消えれば天狗になり自滅するし、前者が消えれば人が集まらず目的は達成できない。互いに矛盾する2つの自我を常に維持し続けなければならず、だからサイコパスにしか向いていない
12
364
143
3,344
396,472
> at larger scale matches an open-source model trained on 11T tokens with ~10x less data and compute. this is lowkey a huge deal.
I've always believed that intelligence should come from simple, general building blocks that learn their own abstractions directly from raw data; in other words, the bitter lesson. I think we're finally seeing a real step towards AI that can learn from all data. Language modeling on raw bytes is a microcosm of this ideal, and to me it's a central problem in AI: if we can't build LLMs that learn end-to-end from bytes, how will we handle the rest of the world's data that we understand even less? Last year we introduced hierarchical networks (H-Nets) as a first step. Our new version finds up to 3-5x efficiency gains in controlled scaling law experiments, and at larger scale matches an open-source model trained on 11T tokens with ~10x less data and compute. The posts below walk through what we found and why we think it works.
3
23
1,261