mathematics Thief & Chef "through the window of differential equations, mathematics sees the light in the real world" / "通过微分方程的窗子,数学家看到现实世界的光" (Jiang Zehan)
Seattle
Joined June 2008
- Tweets14.9K
- Following637
- Followers4.4K
- Likes6.8K
Pinned Tweet
“This Is Not How Mathematicians Are Trained”
数学家不是这样训练的
1, "Solving the forward problem (positive time) and the inverse problem (negative time) together has always been a desire of mathematicians, but they’ve never known where to begin. Neural networks, however, tackle this problem naturally" (将正时间的正问题与负时间的反问题同时求解，一直是数学家的梦想，但他们始终不知道从何入手。而神经网络却能自然地处理这个问题)
2. “Variables, coefficients, even coordinates, are changing. Everything is in flux. This is not how mathematicians are trained. It would be impossible for mathematicians to come up with such a design” (变量、系数，甚至坐标都在变化，一切都处于变动之中。数学家不是这样的训练。这样的设计，不可能出自数学家之手)
3. “Mathematicians tread carefully around composite functions with more than two layers, wary of the many pitfalls, yet neural networks solve them effortlessly, almost nonchalantly.” (数学家在处理超过两层的复合函数时格外谨慎，警惕其中诸多陷阱, 而神经网络却几乎漫不经心地轻松应对)
4. “Neural networks have stacked covers, mathematically speaking, whereas the Numerical Manifold Method typically uses only 3 to 4. In contrast, neural networks stack hundreds or even thousands of such covers. I never imagined anyone would take it that far”. (神经网络在数学上拥有堆叠的覆盖层，而数值流形通常只有三到四层覆盖。相比之下，神经网络则堆叠了上百乃至上千层。我从未想过有人会将其推进到这种程度).
That was what Gen-Hua Shi (石根华) told me after returning from a two-week vacation in early June 2024..
see rest of the story, click the link
open.substack.com/pub/deepma…
#DeepManifoldInterpretation
This paper findings show that Transformers and SSMs can have very different global activation geometries while preserving similar local relationships, including in their late-layer representations.
This is expected in Deep Manifold Theory: different architectures can develop different Neural Fixed-Point Fields (NFPFs), yet produce similar local activation geometry with global context when they learn similar data relationships and achieve comparable performance. Activations are physical covers, the outputs of the NFPF, not the NFPF itself.
1/ 🎉 Excited to present at #EMNLP2026 🎉
Transformers and Mamba are fundamentally different. Do they also represent things differently?
Globally, they differ. Locally, they are similar. 🧵
#DeepManifoldInterpretation
Linear probes observe local manifestations of the neural fixed-point field, while circuit tracing follows selected computational relationships. Neither alone reconstructs the underlying manifold.
That is not a criticism of CTA; it identifies the next level of mathematical interpretation.
💥Paper at #EMNLP2026: Probes tell us whether a model has linearly encoded a concept; they don't tell us which features or circuits produce that encoding.
We introduce Concept-Targeted Attribution (CTA), attribution graphs built around a probe's concept direction. 🧵
Can LLMs infer human actions and motives the way humans do?
In our ACL Findings paper led by @ShresthaKaleen and co-authored by Abhinav Gupta, Harish Dukkipati, and Maja Mataric, we examine this question using a validated game theory framework from cognitive science. We find that current LLMs struggle both to recognize statistical patterns in human behavior and to identify humans’ underlying motives.
These findings highlight important limitations in LLMs’ social reasoning capabilities, raising concerns about their ability to accurately simulate human behavior.
Our paper can be found here: aclanthology.org/2026.findin…, and feel free to reach out to Kaleen for further questions!
#DeepManifoldInterpretation
Neural CDE models the observed flow of agent trajectories, but the underlying computational landscape is the Neural Fixed-Point Field (NFPF), developed during training and deformed during inference.
Agent reasoning follows boundary-conditioned, sampled, iterated integral paths through this field. A Neural CDE approximates those paths; it does not recover the field itself.
Trajectory equivalence is not geometric or topological equivalence.
When an LLM agent executes a long task, the hardest question is usually not whether it failed but where. Which step actually broke the run? 🧵
That’s the failure attribution problem, and it’s becoming central to debugging agentic systems. But current approaches have a real cost problem:
1. Prompting-based pipelines, where an LLM reads the whole trajectory and judges each step, are slow and expensive to run.
2. Post-trained attribution models need failure trajectories labeled at the step level, which are painful to collect and hard to scale.
We think a practical failure attribution model should be lightweight and shouldn’t need step-level labels on failures at all.
So in our #neurips2026 paper, “Tracing Agentic Failure from the Flow of Success,” we ask: can you find where an agent went wrong by learning only what going right looks like?
Meet OAT 🌾
The idea: treat failure attribution as one-class learning. We train only on successful trajectories, using neural controlled differential equations to model how successful runs evolve in latent space. Think of it as learning the “flow” of a healthy agent trajectory.
At inference, we take a failed trajectory and score every step by how far it drifts from those learned success dynamics. Steps that deviate most get flagged as the likely error steps. The algorithm doesn't rely on any step-level annotation.
What we found:
⚡ With just 100 successful trajectories for training, OAT runs 200-5000× faster than prompting-based baselines (including GPT-5 judge).
📈 It also outperforms them on accuracy: about +20% F1 score.
So you don’t have to trade cost for quality here. You can get both.
Why I’m excited about this: successful runs are cheap and plentiful, while carefully annotated failures are not. Building diagnostics on top of the data we already have feels like the right direction as agent systems grow longer and more complex.
Huge thanks to my amazing student @Samuel861025 for leading the effort, and collaborators from Microsoft Research @ShaleenDeep and Yiwen Zhu.
📄 Paper: arxiv.org/abs/2607.12747
Goodfire’s Monitoring and Discovering Reward Hacking with Internal Representations during LLM Evaluations (Leon Bergen et al., arXiv:2609.19101) demonstrates that reward hacking produces detectable signatures in neural activations.
From the Deep Manifold perspective, reward hacking reflects a deeper structural source of misalignment. The propertyless, full-rank relational structure of neural networks enables powerful connections, including between semantically unrelated representations.
This flexibility can also enable unintended computational shortcuts that satisfy reward conditions while violating task objectives.
Such shortcuts can be interpreted as misaligned trajectories through the learned neural fixed-point field (NFPF).
Activation probes observe these trajectories through physical covers, but detection does not necessarily correct the underlying mathematical covers.
What makes AI powerful is also an inherited source of misalignment.
very good questions...
“Completely break our intuition”?
Yes, if we still view neural network geometry through the lens of Riemannian manifolds.
No, if we view it through the lens of stacked, piecewise manifolds.
xitter.cf/BetaTomorrow/status/20…
one of the strangest things about deep learning is that high-dimensional spaces behave in ways that completely break our intuition.
1. in high dimensions, almost all the volume of a sphere concentrates near its surface. weird, right?
2. randomly chosen vectors become almost perpendicular to each other as dimensionality increases.
3. in thousands of dimensions, two random vectors will typically have a cosine similarity close to zero.
4. neural networks can use these high-dimensional spaces to represent different features as directions in their activation space, without necessarily making them overlap too much.
5. but features can still interfere with each other and share dimensions. this is one reason understanding what a neural network has actually learned is so difficult.
6. gradient descent doesn’t just adjust weights. through training, it also shapes the geometry of the representations the network learns.
To channel my inner Schmidhuber, we found this softmax "bug" ~1.5 years ago and published in this paper arxiv.org/pdf/2603.11682
In reality, it's pretty hard to come up with a training scenario where this matters because gradient w.r.t. highly likely tokens is tiny anyway. Nonetheless I recommend fixing it -- one less thing to worry about! Check out @jon_barron 's work for an elegant fix!
#DeepManifoldInterpretation
LEAP learns stochastic action over complete trajectories, rather than local transitions.
Deep Manifold reading: Training develops a Neural Fixed-Point Field with stochastic fixed-point classes.
Stochasticity perturbs trajectories within or between these classes, while boundary conditions shape the realized path.
Inference is an iterated integral through the learned field.
@liaw_sarah, @YuanqiD, @du_yilun
We introduce action learning: learning a stochastic action that scores how plausible an entire trajectory is, directly from observed trajectories.
No known forces. No potential energy function. No simulator likelihood.
We think this is a new paradigm for reasoning about trajectory data.
Joint work with @du_yilun and @YuanqiD.
🔥 Recovers underlying physical dynamics with over 2,000× lower error than local regression in a double well system.
🪐 Identifies a hidden 0.01% perturbation in the Sun's mass from trajectories of a 31 body orbital system.
🧬 Reuses a single learned molecular action for trajectory completion, steering, constrained optimization and temperature transfer.
The core idea: One learned action → reuse for many possible downstream tasks!
#DeepManifoldInterpretation
An excellent paper showing that some neural network training instabilities are not intrinsic to learning, but artifacts of numerical computation in softmax and cross-entropy.
From a Deep Manifold perspective, numerical errors can distort the development of the neural fixed-point field, creating artificial jaggedness and disrupting residual-energy dissipation. Not all observed instability is geometric instability.
#DeepManifoldInterpretation
Paper: CulturalRiddles: A Multicultural Benchmark of Riddles
** What makes AI powerful is also a source of misalignment **
Misalignment is a model or computational error, and its root causes are inherited by design, mathematically:
1. Inverse learning
2. Propertyless and full-rank relational structure
3. Data complexity and numerical approximation
The difficulty is that these are also what make AI extraordinarily powerful.
We therefore face a double-edged problem: the mathematical sources of capability are also inherited sources of misalignment.
On top of that, MoE architecture and some current training practices can further amplify these inherited errors, making alignment even harder.
xitter.cf/BetaTomorrow/status/21…
“Knowing a language without its culture is like reading a restaurant menu without ever tasting the food: you can name every ingredient, yet miss what the meal means to the people eating it.”
In this blog, Vinod Anbalagan, Tianyi Hu, Madeline Smith, Julia Kreutzer and some of the research leads on CulturalRiddles, share the story and key findings behind our benchmark exploring what AI models miss when it comes to cultural understanding. cohere.com/research/papers/c…
(Key takeaways below 🧵)
#DeepManifoldInterpretation
The paper uses a flow-matching residual as a surrogate for action likelihood, not an exact probability density. Its value anchor is therefore essential to align the resulting Q-value with Bellman targets.
In Deep Manifold terms, this suggests a useful hypothesis:
Richer relational supervision can improve the consistency of a learned neural fixed-point field, but better reward prediction alone does not prove that the field is smoother or homologically more stable.
That distinction would require additional experiments measuring sensitivity to boundary-condition changes, numerical residuals, and trajectory stability.
GVC provides evidence that learning relationships among states, actions, and returns can outperform direct value regression. Its contribution is consistent with the Deep Manifold emphasis on relational learning and numerical constraints, without directly validating fixed-point-field theory.
@alexjswerdlow, @perryadong, @chelseabfinn
In language models, turning a generative model into a critic is simple: pass in the output and prompt for a score. Robot policies can't do this - they output physical actions & there's no built-in vocabulary to predict scores.
So when it comes to RL on robot policies, we typically train a separate critic from sparse rewards - blind to everything the policy learned from the actions in the pretraining data.
We think this has it backwards - can a critic learn from those actions the way the policy does? (1/n)
#DeepManifoldInterpretation
a good work !!!
@ilzhechev, @RadoslavRalev,@srijithrajm,@jainhom,
@tchutch94, @simba_khadder
Happy to share that our paper, “The Geometry of Agent Memory,” was accepted as a poster at the PALM workshop at NeurIPS 2026.
Paper: arxiv.org/abs/2610.26433
Project page: izhv.eu/agent-memory-geometr…
#NeurIPS2026 #PALM #AgentMemory #AIResearch
#DeepManifoldInterpretation
From the Deep Manifold perspective, the effectiveness of Looped Transformers can be explained by two complementary mechanisms:
1. Fixed-point normalization — Successive iterations progressively align the computational state toward a stochastic fixed-point class.
2. Dynamic inertia (residual energy) dissipation — Successive iterations progressively dissipate residual energy, improving stability and convergence.
Together, they describe why looping can be effective beyond simply increasing computational depth.
3 weeks, and the cache climbed up from the bottom to the top ;) 🚀
Now, this is what I call upward mobility
arxiv.org/abs/2610.02383
#DeepManifoldInterpretation
Misalignment is a model or computational error, and its root causes are inherited by design, mathematically:
1. Inverse learning
2. Propertyless and full-rank relational structure
3. Data complexity and numerical approximation
The difficulty is that these are also what make AI extraordinarily powerful.
We therefore face a double-edged problem: the mathematical sources of capability are also inherited sources of misalignment.
On top of that, MoE architecture and some current training practices can further amplify these inherited errors, making alignment even harder.
xitter.cf/BetaTomorrow/status/21…
Excited to share our 𝗖𝗶𝘁𝗲𝗔𝗻𝗰𝗵𝗼𝗿 project led by @cytong77! Tracing a hallucinated citation back to its likely source paper makes the assessment more explainable and convincing.
Paper: arxiv.org/abs/2610.30355
Code: github.com/oldwizard7/CiteAN…
Dataset: huggingface.co/datasets/chen…
#DeepManifoldInterpretation
We need more such papers and research on blue-collar AI for the enterprise, with high precision, like accuracy over 98%.
📢 Introducing WorkForge: a scalable framework that constructs verifiable training environments for long-horizon work agents from real-world resources.
Starting from expert workflows, it builds authentic workspaces, extracts checkable facts, and derives tasks plus dual (programmatic + semantic) verifiers grounded in observable evidence.
🚀 We scale to 16.7K environments across 40 professional domains covering 60 file types, enabling consistent gains on long-horizon work benchmarks as data volume and interaction horizon increase.
📚Paper: arxiv.org/pdf/2610.04906 @zhang_leo54389 @ZoeyyyZhououou @Be1ong1
#Tencent Hy#AIAgents #WorkAgents #AgentTraining
