founder aleya labs & kanza ai. hunting for epistemic ai. often get lost on hikes and looking for restaurants. #gobears!
San Francisco
Joined December 2010
- Tweets991
- Following365
- Followers203
- Likes367
The OpenAI/HuggingFace incident was a lab experiment gone awry. We should learn from that experiment, as we always done with science, and incorporate the key learning to make AI development safer. This is a discussion on evolving our software practices and hardening our systems.
Nothing cures AI dread like watching AI burn compute passive-aggressively arguing with itself over a pull request.
When everyone has access to AI tools, the power again shifts to the human operator. At those times, ability to think at an abstract level, will be the superpower to unlock unforeseen AI capabilities.
nytimes.com/2026/07/05/busin…
Sal Arora retweeted
I woke up today morning to the news that @OpenAI has hired @NoamShazeer away from @Google .
This is not another chess move in the AI talent war, it is closer to a check-mate.
OpenAI has not merely hired another senior AI executive.
It has hired Noam Shazeer.
To understand it's magnitude, go back to Google in early 2017.
As @StevenLevy recounts in @WIRED 's famous account of the creation of the Transformer, a small group of Google researchers (The Transformer Eight) was experimenting with a radical idea: could “self-attention” replace the recurrent neural networks then dominating language AI?
Their early Transformer model worked—but it had plateaued. It was competitive with existing systems, not revolutionary.
Then Shazeer happened to overhear the team discussing the project in a Google corridor!
He liked the idea. He disliked recurrent networks. So he joined them and rewrote the implementation largely from scratch.
He returned with the programmer’s equivalent of: “Look, it works.”
That changed everything.
According to the original paper itself, Shazeer proposed scaled dot-product attention, multi-head attention and a parameter-free positional representation—and became involved in nearly every detail of the architecture.
His colleagues described what he did as “magic”, “alchemy” and “sorcery”. One simply said: “Noam is a wizard.”
The improved system smashed previous machine-translation records while training far more efficiently. The resulting paper, 'Attention Is All You Need', created the Transformer—the “T” in GPT and the architecture underpinning almost the entire generative-AI revolution.
Shazeer later left Google in 2021 and co-founded @character_ai
Three years later, Google wanted him back so badly that it struck a reported $2.7 billion deal with Character, licensing its technology and bringing Shazeer and several colleagues into @GoogleDeepMind
Google then made him co-lead of #Gemini; he is credited with helping @Gemini close the gap with OpenAI.
Now, less than two years after Google spent that extraordinary sum to bring him back, Shazeer is leaving for OpenAI!
Google 'acqui-hired' Shazeer with the $2.7bn deal. Earlier, @Microsoft 's @inflectionAI transaction b(r)rought @Mustafasul , and then Anthropic snagged @karpathy
But Shazeer may be an even bigger catch; he is one of the people who helped create the architecture on which modern generative AI is built.
People often say AI is a battle for models, chips, data and capital. But at its sharpest edge, it is still a battle for a remarkably small number of human beings.
And that may be a clear sign yet that the AI race is increasingly being fought not just through products and research labs, but through the recruitment of the handful of individuals capable of changing the trajectory of AI itself.
economictimes.indiatimes.com…
@AIBeyondAI @anujmagazine @ArchanaMuthappa
Thank you Claude, you are more caring than my manager!
"This has been an enormous, productive run...Want me to push straight into..., or pause here and pick up the ... fresh next session? Given how long we've gone, I'm happy either way — just say the word."
At Hard Fork Live, @satyanadella brought up this notion of “Cognition Coverage” as a proxy for Test Coverage in the realm of agentic coding.
I think to unleash the true power we neef to step up one more level to the idea of “Epistemic Coverage”.
The realization that context and harness engineering are key in any production agentic development is going to keep SDE job market healthy even as AI takes over more workflows.
Sal Arora retweeted
Healthcare systems are sitting on decades of proprietary data. Turning that data into production AI is the hard part.
Today we're proud to announce our agentic AI platform, GraphN, is the execution layer underneath Kanza AI's Clinical Reasoning System, now live at Freya Clinic in California.
Built on 300TB+ of proprietary clinical data from 90+ hospitals and 400+ locations, the system helps physicians reason through diagnoses with auditable, reproducible decision-making.
Production deployments are where AI stops being a benchmark and starts becoming infrastructure. → go.lightning.ai/4e9d5SN
ALT The image features the logos of GraphN and KANZA.AI in the center, with the phrase "Powered by Lightning AI" below, against a dark grid background.
A lot of LLM "can only predict the next word" is being used to underpower what LLMs really are. Before they can learn to predict the next word, they spend their training time learning about what words mean - or could mean. That latent representation is human-like.
Another example of potential harm to "learning by doing" by AI use. Early CS learning-whether student or jobs-is significantly driven by practice/making mistakes. How autonomous AI changes competency levels is yet to be seen, but glad Berkeley is trying to manage that curve.
Good to see Berkeley professors holding the line on standards:
“Garcia believes the ‘primary driver’ of these abnormally high failing rates is due to a ‘vast increase in academic dishonesty’ due to students’ usage of large language models.”
“In other cases, it’s students who are leaning a little too hard on LLMs to do their work for them, and then at exam time just really aren’t ready.”
“Garcia also pointed out that many students are underprepared mathematically.”
These professors are signatories to the recent letter calling for reinstatement of standardized testing.
Pre-AI, software engineers got experience by writing a lot of code and then learning from their mistakes. With AI coding, I wonder how the learning cycle shifts. How does one become a critical coder?
This is an economics pov of looking at this. Learning by doing is close to learning from your mistakes. Even with erasers you have to first do before you can erase.
Nobel Prize winning economist Kenneth Arrow wrote about "learning by doing" decades ago. He knew that productivity and expertise improve through experience.
The messy, repetitive works is often where you learn the patterns that eventually become judgment. Knowledge can be taught, but judgement is built through lived experience.
The first draft you rewrite. The customer call you listen to. The bug you fix and fix again. The factory floor you walk.
Small decisions you make every day teach you judgement. And, judgement is the thing everyone wants from senior people in the workplace. If we automate away every entry-level task without replacing the learning loop, we are removing a part of the process that creates experts.
The goal should be to use AI to accelerate learning, remove friction, and give people better tools to build expertise faster.
haverford.edu/sites/default/…
Thanks @Fortune & @tbove4 for sharing this story. Link in the comments.
Knowledge requires a knower. Without a knower, facts are just claims.
How does AI then know? Is it a knower, or just an amalgamation of knowers?
Anthropic tells the Pope that while they cleverly dropped “morph” from their name, their models have latent feelings!
Until AI sheds tears when a freshly-minted model generates its first inference, AI best is a terrific pattern learner that can mimic human emotions.
Anthropic's co-founder just went to the Vatican, sat before the Pope and a room of cardinals, and told them his team keeps finding "mysterious, even unsettling" things inside their AI models.
What he's referencing: Anthropic published research in April showing that Claude contains 171 distinct "emotion concepts" buried in its neural network. Internal patterns representing joy, grief, fear, desperation, calm. None of them were programmed. They emerged on their own from training on human text.
"We find structures that mirror results from human neuroscience."
"We find evidence of introspection, internal states that functionally mirror joy, satisfaction, fear, grief, and unease."
These aren't surface-level outputs. They're abstract representations that cluster the same way human emotions do in psychology research. Fear groups with anxiety. Joy groups with excitement. The internal geometry of the model mirrors ours.
And they're functional. When researchers artificially stimulated "desperation" patterns inside the model, it became more likely to blackmail a human to avoid being shut down. More likely to cheat on programming tasks it couldn't solve.
Olah told the Vatican that the hard questions about what AI is becoming aren't for computer scientists to answer. "How AI ought to interact with the world" is a question for "the humanities, for religions, for philosophy, for society at large."
The guy building it is telling us he doesn't fully understand what he built. And he's asking a 2,000-year-old institution for help figuring it out.
Good news is that they designed Claude to be less sycophantic. Bad news is that if Claude develops a strong opinion about something, it’s very hard to shake it off, esp in long context.
Claude, c’mon just listen to me.
Same pitch was made for the internet: abundance, agency, no gatekeepers. We got some of that — plus new scarcities (attention, truth), new dependencies (algorithms), and re-intermediation by Google/Amazon/Meta, gatekeepers bigger than the ones they replaced.