RL for domains where human judgment is critical. AI Lead/Cofounder @tacitco. Father. Epeeist. Rower. Gardner. Reader. I walk a lot and I train a lot.
vim+tmux ∪ New Zealand
Joined March 2007
- Tweets17.5K
- Following1.4K
- Followers1.9K
- Likes7.4K
Pinned Tweet
In statistics, the concept of "Four Moments" allows us to understand the shape of the data without storing every data. Here is a good overview: youtube.com/watch?v=SZ3T1cSX…
Now, how many data labs/providers care to profile the statistical characteristics of their data to ensure their data are a) totally synthetic; b) 100% inline with real world distribution?
Now, imagine you can do "Four Moments" over a concept in latent space, not just a single variable.
Alex Dong retweeted
Fukuyama on meeting Qaddafi: "He had a zero-sum view of wealth: wealth could not be created, only taken from someone else. I asked him then where Bill Gates—at that time one of the richest men in the world—got his money. Qaddafi said he stole it from his customers. I pointed out that they got useful software in return; he thought about it a bit and said that Gates stole it from the companies that made computers." richardhanania.com/p/cosplay…
To anyone who is interested in history, how would you rate Kadar's legacy?
en.wikipedia.org/wiki/J%C3%A…
Alex Dong retweeted
Well, it depends on what you mean by best code. Is it:
- The most reliable code I've seen?
- The most necessarily complex code I've seen?
- The smallest elegant solution I've seen?
- The best fix while debugging?
All of those would have different answers. DaveC writes some mighty reliable code without making it look complicated. RaymondC can debug vintage apps like no one else. The Kernighan identity for counting bits (n & (n-1)) is pretty genius. And so on.
It's like most things — there's no "one" smartest guy. Who you'd call depends on the problem you're facing.
Alex Dong retweeted
Bill was NOT the best programmer of his generation.
I've read his code. I've read Paul's and Monte's. And the reality is that the company's original crown jewel, the BASIC interpreter, is, in fact, pretty genius. Especially for young guys.
But it's not the best code ever written, or the best code I've ever seen, or anything like that. It's good, solid assembly code written by kids to pack a LOT of functionality into 8K. Code to be proud of.
BASIC didn't establish Microsoft's foothold because it was new, unprecedented, or even that hard to do. It was the right solution at the right time in the right place.
The hardest parts - the floating-point math - were heavily inspired by the DEC Focal interpreter, to which they had source code access.
And they had access to other DEC interpreters to see how they worked. None of this was rocket science. But it was something Bill was very good at: fast following.
That's a real skill, and it shouldn't be minimized. It doesn't matter what you can ship 3 years from now. It matters what you can ship 3 months from now.
That sense of angst and dread that likely drove them - afraid that someone else would get there first - that was the key battle. Not the technical side.
Not "how will we ever write this" but rather "how will we write this before someone else does it".
And they were first, and it was good, and the rest is history.
Alex Dong retweeted
I wanted to understand the structural and philosophical differences between GPUs, TPUs and newer alternatives (like Groq LPUs) without getting drowned in specs and jargon.
Mispronunciation of the day: "soldered" 🤦♀️...
0:00 Intro
0:56 The origin story of TPUs
3:42 The first TPU: an ASIC for matrix multiplication
5:21 SRAM vs DRAM
7:21 The memory bandwidth problem
9:14 Systolic arrays
12:48 TPU vs GPU
17:51 Groq’s LPU
25:38 GPU vs LPU
28:36 NVIDIA on the AI chips boom
Alex Dong retweeted
The US is in deep trouble if we ever have a big extended downturn. Other nations are more used to resilience in defeat, while we tend to fall apart and quit.
"Among prisoner of war communities, the prisoners who behave worst are Americans." Kurt Vonnegut, 1970. #DeadWrongHistory
`=SKEW(...)` and `=KURT(...)` in Excel if you are wondering.
I'm not going to tell you the latent space four moments because .... well, i don't know yet.
Alex Dong retweeted
Almost all the great people I work with have side projects.
Only a minority of those are code. People are woodworking, metalworking, sewing, gardening, sports coaching, building planes, building boats, and a ton more.
Side quests are a good sign, but often aren't visible in GH.
Every time I looked for a decent benchmark, I got bitterly disappointed.
It's the same feeling when you look at the code of some popular open source project's github repo.
🫣
1. Tried to cancel Loom.
2. Can't find the cancel button. But a tiny copy suggests that I "downgrade" to Starter Plan;
3. I downgraded
4. Got email congradulating me on "14-day free-trial Pro Plan";
5. I handed this bullshit to codex.
6. ....
7. Sorted.
WTF, @Atlassian?
“Cut once. Cut deep. “ is the old PE motto.
The thing is, this is not going to be only once. And it is not going to be deep enough.
Eg. Twitter is run by 10-15% of its original workforce.
Alex Dong retweeted
if anyone finds this compelling I recommend the sport of epée fencing, which is basically just this with more complexity
My wife and I do rock paper scissor to decide stuff a lot, and let me tell you, that game has surprising depth when pushed by two nerds who are determined to win. I have long randomized sequences memorized to throw her off, and we know the conditional probabilities of (naive) follow up moves, and psych each other to push the other to become more predictable and naive. We often go 5, 6 rounds of both mirroring the other before someone outsmarts the other. Everything has more layers than you’d naively assume.
Alex Dong retweeted
In a weird way, everybody is rooting for you. When you go to a job interview, the interviewer wants you to be amazing so they don’t have to look anymore. when you go on a date, the person wants to have an amazing date with you. If you make a piece of content, people want to see amazing pieces of content and they have no problem if it’s yours.
the instant you get into the mindset that people want you to do well, your entire world gets better. And you’ll be shocked to know this pattern keeps repeating in different areas of life.
Alex Dong retweeted
i've reproduced the results from the MIT blog post today on weight-space programming. but one additional result i have to add: this is a new, step-change emerging capability that's appeared in just the latest set of models.
Ellison admires Japanese culture so much that his residence in Woodside has a perfectly designed japanese garden.
ORACLE'S BILLIONAIRE BET: Larry Ellison Personally Brokers Multimillion-Dollar Deal to Bring Legendary Computer Scientist Yoshihiko Futamura to Oracle Labs to Revolutionize GraalVM
In a stunning move, the Oracle founder reportedly goes straight to the source of partial evaluation, betting millions on the Japanese computer science pioneer whose ideas could reshape the future of software.
I am going to steel-man for the juniors, even though I share the sentiment.
1) they seem to grow up in a perpectual fear response state. Global warming is going to kill you; now AI is going to steal your job, then kill you. We know it's an exaggeration but they don't. The old world order is gone but there isn't anything to replace it. We are living in an uncertain time. We have more fat/cushions but they don't.
2) In my first job, I was naive enough to take on a super tough technical job. (Intel's Xeon chip couldn't be sold in East Asia because of a bug in their driver. No one could fix it.) I was given free rein for more than 3 months. In today's world, I suspect no employer would be willing to do that level of investment or trust.
3) (extension to the prev. point) the germination period for a person to become a productive member of the society has exploded. yet the education system is so backward-thinking that kids are not given the right prep.
Case in point, I had a junior colleague who is timid, very green and does irresponsible vibe coding.
Only a few months ago did I find out that he was actually a member of an elite unit in IDF. So he is clearly capable but I didn't see that at all. I reflect on that a lot.
At the end of the day, at least to me, the purpose of life is to help people around me to flourish, and THAT certainly includes mentoring/guiding those junior members of our society.
Heard a story over the weekend where a friend asked junior team members to use AI assistants to deliver a first draft of a client presentation. They were surprised at the results they got back and asked to see the chat conversations.
What they found out was that the junior employees had just thrown the email into a chatbot, didn't do any additional turns, and simply turned in the one-shotted result.
I feel like a lot of junior folks don't realize that not only are they totally replaceable by AI at this point, but I personally think if you use them and they're performing like it's still 2022, you're actually paying money to slow yourself down lol. I would not hire any junior people who haven't figured out how to work with AI, which is the bare minimum in 2026.
If they need to be babied and shown how to do everything, I don't know what their advantage is of being young. If folks like me in our 40s are spending all our time testing out new technology and orienting our work towards it, what is their excuse? They don't have years of ingrained habits.
Kid. There is no such thing called “crusty senior engineer”. I know what you mean though, even though the agism undertone is offensive.
There is only good engineer vs. bad engineers. Everything else is just fluff and bulls.
Btw. How old is Elon again?
The dirty little secret to SpaceX’s success was never that they dispensed with all the crusty senior engineers at NASA and legacy aerospace in favor of hot shot 24 year olds. It was that they hired every single one of those crusty old senior engineers and dispensed with the broken managerial culture in big corporate aerospace. They got where they are by transplanting Prime talent into a Founder corporate stucture.