Founder at Noumenal Labs. Building Physical AI.
Dallas, TX
Joined May 2008
- Tweets4.8K
- Following1.4K
- Followers1.3K
- Likes7.3K
Jason Fox retweeted
Our friends @noumenal_labs strapped an XTR-0 to an agriculture bot and used it as the brains 😁
Not how I imagine how people would use the desktop form factor but very cool!
This is why we are creating Z1 m.2 sticks, which will be far more compact and 100k× more pbits than X0
Jason Fox retweeted
During this year’s NeurIPS afterhours, we’re hosting an intimate gathering of researchers, founders, and investors exploring the intersection of computation, thermodynamics, and embodied intelligence.
If you’re working at the edge of alternative architectures, stat-phys-inspired ML, or embodied intelligence, you’ll feel right at home.
What to expect:
• Thought-provoking conversations on alternative compute paradigms
• A curated group of technologists & builders
• Great food, great drinks, great company
Hosted by cyber•Fund, Noumenal & collaborators. Registration & approval required: luma.com/tqgzktjg
Looking forward to connecting with the people shaping the next chapter of intelligent systems.
@mjdramstead @cyberfund
Jason Fox retweeted
New blog post by @noumenal_labs: “WTF is the FEP? A short explainer on the free energy principle”: noumenal.ai/post/wtf-is-the-…
Really happy to share this one! We discuss the free energy principle: What it is, what it is not, what promise it holds, why it can be extremely useful, and why it has yet to live up to the hype.
Jason Fox retweeted
New blog post by @noumenal_labs: “Grounded rewards in the era of experience: A commentary on ‘Welcome to the era of experience’”: noumenal.ai/post/grounded-re…
Here’s the tl;dr:
• This post is a commentary on a new paper by Silver and Sutton, entitled “Welcome to the Era of Experience” (2025).
• Silver and Sutton (2025) provide a thought-provoking discussion of the last decade of research and development in the field of artificial intelligence (AI), and where the field is heading. The core idea is that we have reached a performance ceiling for AI agents trained via supervised learning from human data — and that we have entered a new epoch in the development of AI, which the authors call the “era of experience.”
• The era of experience, as the authors describe it, is a forthcoming phase in the development of AI that will be characterized by “grounding” in the real world, online action-perception loops, physical embodiment, environment-sourced reward signals, and online real-time experiential learning.
• In particular, Silver and Sutton argue that the era of experience heralds a shift from hand-crafted, user-specified reward functions and the heavy use of human expert feedback and supervision, towards “grounded rewards,” which are measured and evaluated by AI agents themselves by continually assessing the sensory consequences of their actions in real time.
• Here, we review and evaluate their argument. We enthusiastically embrace several aspects of their discussion and offer some constructive feedback pertaining to the learning of grounded reward functions.
Jason Fox retweeted
Neuroscientist Dr. Jeff Beck from @noumenal_labs discusses the fundamental nature of representation, understanding, and modelling, comparing biological intelligence with current artificial intelligence.
Jeff argues that *how* information is represented dictates predictive ability and that LLMs, while impressive at symbol manipulation and pattern matching (like next-word prediction), lack the *grounded*, causal understanding of the world inherent in biological systems.
Timestamps:
00:00 - Cat visual cortex experiments & discovering orientation sensitivity (slide projector analogy)
01:49 - Representation choice and neural coding (orientation vs. feature intensity)
02:30 - Choice of representation impacts predictions; generative models
03:15 - Importance of choosing the right generative model for predictions
03:35 - The problem: We don't know the brain's true generative model
03:55 - Theory of Mind (ToM) in LLMs
04:05 - Jeff Beck's ToM tests on early ChatGPT (stapler example)
05:40 - ChatGPT recognizing the ToM test vs. passing it
06:32 - Analogy: LLMs recognizing known problems vs. generalizing (sum/product riddle)
07:25 - Do LLMs implicitly build world models? Vicarious experience analogy
07:59 - The difference: Grounding symbols in reality outside language
08:35 - AI Alignment: Difficulty in capturing human reward functions & belief formation
09:21 - Nightmare scenario: Humans as "complacent value function selectors"
09:44 - Hope: AI enhancing human understanding, not replacing thought
10:08 - Philosophy of science: Science realism vs. modeling pockets of regularity
10:39 - Noise in models as ignorance or deliberate exclusion (design choice)
11:00 - Design choices in science, controlled experiments, and induced bias
11:29 - Are there true, discoverable mathematical laws of the universe?
11:41 - Is there a "true" ground truth distribution (P)? Beck's answer: No (with nuance)
12:55 - Ontological vs. Epistemological divide: Perfect models vs. models of regularities
13:21 - Are scientific models "false by definition"? The Bayesian perspective
14:07 - "All knowledge is conditional"; Are foundational theories (e.g., FEP) true or just perspectives?
14:51 - FEP as a mathematical framework, not a theory; models are just models
15:54 - Legibility vs. Utility: Useful but illegible AI models
16:01 - Prediction vs. Explanation: Trusting black boxes can be unsatisfying
16:30 - Why understanding AI matters: Ensuring alignment with human decisions/values
17:08 - Line-of-sight legibility as an alignment approach
17:14 - Benefits of explainable AI: Human understanding and value alignment verification
18:21 - RL components: Prediction engine, reward function, policy; the alignment challenge
19:25 - Trusting AI = Trusting its policy aligns with our reward + its superior beliefs
19:57 - Language: Intrinsic representation vs. pointers between shared minds
20:21 - Why language works: Shared internal models and common grounding
21:00 - Basis of shared understanding: Not linguistic, but shared experience/intuitive physics
22:44 - Consciousness and language as lossy, simplified summaries of complex brain processes
23:22 - Evidence for simplification: Brain regions, perception vs. representation; limits of language models
24:32 - Counterpoint: Language captures complex/ambiguous human concepts
24:54 - Language as massive compression: The information bottleneck (Meister's paper)
26:22 - Implication: Language/actions are poor representations of internal understanding
27:03 - Can language models understand? The mimicry argument (Piantadosi)
27:33 - Beck's skepticism: LLMs excel at prediction/mimicry, not true understanding
28:09 - LLM explanations replicate structure but lack grounding
28:54 - Beck's test for LLM understanding: Genuine novelty beyond training data
29:19 - Summary: Symbol manipulation is not understanding; grounding is key
30:06 - Abstraction and Idealization in scientific modeling ("The Brain Abstracted")
30:45 - Revisiting Newton: Intuitive physics is correct for our world; idealizations are simplifications
32:01 - Sophistication & boundaries: Nested systems vs. one complex system?
32:32 - The boundary problem in FEP/Markov Blankets: Where to partition?
33:41 - Beck's research: Finding principled partitions based on interaction dynamics
35:44 - Beyond direct experience: Imagination, language, and learning
36:16 - Human creativity: Creating new *things* by combining modeled objects (Systems Engineering)
37:43 - Goal for AI: Automating systems engineering for creative combination
38:17 - Sutton's "Reward is Enough" paper
38:25 - The challenge of "Reward is Enough": Defining and obtaining the *right* reward function
39:02 - Difficulty of eliciting individual reward functions
39:52 - The core alignment problem: Accessing and representing individual reward functions
40:13 - Impossibility: Disentangling beliefs and rewards from observed actions
41:51 - Argument analogy: Disagreements stem from different beliefs or values
43:00 - Prerequisite for value inference: Understanding belief formation
43:13 - Building aligned systems: Sparsity of data, meta-models vs. base system modification
43:46 - Proposed solution: AI layer that models the human's belief formation system
44:40 - Alignment process: Align beliefs first, then address value differences
45:00 - Conclusion
CC @mjdramstead
Jason Fox retweeted
New blog post by @noumenal_labs: “AI alignment and theory of mind”
noumenal.ai/post/ai-alignmen…
Here’s the tl;dr
• In this blog post, we discuss the approach to AI alignment pursued by Noumenal Labs
• The goal of AI alignment is to develop methods and protocols that ensure that artificial agents make decisions and execute actions that are consistent with the goals and values of humans. Technically speaking, this means designing machine intelligences that share a policy or reward function with their human users.
• This is an important goal. But state of the art approaches in artificial intelligence are not designed in a way that can deliver on these promissory notes.
• There is no normative solution to the problem of reward function selection. Expert trajectory replication, reinforcement learning with human feedback (RLHF), and the automated generation of reward functions from the stated preferences of users are not viable solutions to the AI alignment problem.
• The missing ingredient for AI alignment is the capacity to take the perspective of another and evaluate their beliefs — what is known as theory of mind.
Jason Fox retweeted
New blog post by @noumenal_labs: “Filling the gaps in active inference”
noumenal.ai/post/filling-the…
Here’s the tl;dr
• In this blog post, we discuss active inference. Specifically, we discuss key gaps in state-of-the-art applications of active inference in artificial intelligence — and how Noumenal Labs is working to fill them
• Active inference is a form of Bayesian machine learning that replaces the objective function used in traditional machine learning — the cost or reward function — with an information theoretic surprise minimization objective, which is used for both inference and learning.
• While theoretically well grounded, active inference has yet to yield state of the art performance in practice.
• This gap in performance is largely due to the relative simplicity of the generative models, underlying state space, and relationships typically used in the active inference literature. Generically, these models lack the requisite structure to represent the world as we understand it, i.e., as composed of distinct objects and object types, as well as object-type-specific interactions, unless hand-specified by the user.
• Noumenal Labs is developing a new approach to macroscopic physics discovery which enables us to build machine intelligences that discover the different objects/concepts and object/concept types that generate time series data, as well as the type-specific rules that govern their behavior — doing so directly from data, in an unsupervised manner.
• Noumenal Labs’s approach is to build models that are explicitly structured — in a manner that is consistent both with the means by which we intuitively understand and conceptualize the world, and the means by which our best scientific models describe it.
Jason Fox retweeted
I’m happy to share the new @noumenal_labs preprint, “Dynamic Markov blanket detection for macroscopic physics discovery,” by Jeff Beck and me
arxiv.org/abs/2502.21217:
Here’s a thread!
This paper resolves a key outstanding issue in the literature on the free energy principle (FEP), namely, to develop a principled approach to the detection of dynamic Markov blankets. The FEP has been proposed as a generalized modeling method capable of mathematically describing arbitrary objects that persist in random dynamical systems; that is, a mathematical theory of “every” “thing.”' The FEP starts with a mathematical definition of a “thing” or “object”: any object that we can sensibly label as such must be separated from its environment by a boundary. Under the FEP, this boundary is formalized as a Markov blanket that establishes conditional independence between that object and its environment.
Nearly all work on the free energy principle has been devoted to explicating the dynamics of information flow in the presence of a Markov blanket, and so the existence of a Markov blanket is usually assumed. Garnering significantly less interest is the question of how to discover Markov blankets in the first place in a data-driven manner.
Accordingly, in this preprint, we leverage the free energy principle (FEP), and the associated constructs of Markov blankets and ontological potentials, to develop a Bayesian approach to the identification of objects, object types, and the macroscopic, object-type-specific rules that govern their behavior. This is accomplished by reframing the problem of object identification and classification and the problem of macroscopic physics discovery as Markov blanket discovery. More specifically, we develop a class of macroscopic generative models that use two types of latent variables: (1) macroscopic latent variables that coarse-grain microscopic dynamics in a manner consistent with the imposition of Markov blanket structure, and (2) latent assignment variables that label microscopic elements or observations in terms of their role in a macroscopic object, its boundary, or the environment.
Crucially, these latent assignment variables are also allowed to evolve over time, in a manner consistent with Markov blanket structure. As such, this algorithm allows us to identify not only the static Markov blankets that have concerned the literature to date, but also — and most importantly — to detect and classify the dynamic, time dependent, wandering blankets that have caused controversy in the literature since the turn of the 2020s. This allows us to model the exchange of matter and energy between an organism and its environment, but at the cost of massively expanding the set of Markov blankets that we need to consider. We handle this additional complexity by applying the automatic Occam’s razor effect of Bayesian inference to select the best partition.
In developing this algorithm, we provide an empirical, definitive conclusion to the debates about whether objects like flames and living creatures, which undergo material turnover, have Markov blankets. We demonstrate that they do, providing simulations that identify the Markov blankets of Newton’s cradle, a burning fuse, a Lorenz attractor, and an artificial lifeform generated by Particle Lenia.
We also make philosophical and theoretical contributions to the free energy principle. We offer a new, deflationary take on the free energy principle, focusing on the role of the modeler. In the extant literature on the free energy principle, the role of surprise minimization is treated tautologically, not empirically: the idea is that “things minimize surprise”. Our treatment focuses more on the pragmatics of empirical modelling, where the role of surprise is better summarized as “labeling something as an object of a specific type minimizes my surprise.”
Jason Fox retweeted
New blog post by @noumenal_labs: “Designing grounded world models via macroscopic physics discovery”
noumenal.ai/post/designing-g… 1/4
Jason Fox retweeted
The Noumenal Labs white paper is now live on arXiv: arxiv.org/abs/2502.13161. Enjoy!
Jason Fox retweeted
We’re very proud to share the @noumenal_labs white paper, “How To Build A Brain”:
noumenal.ai/how-to-build-a-b…
1/7
Jason Fox retweeted
Now hiring! Introducing Noumenal Labs (noumenal.ai/), a new deep tech company focusing on multiscale, multimodal time series modeling, object-centered causal physics discovery, continual learning, and energy-based architectures 1/5
Jason Fox retweeted
New blog post by @noumenal_labs. We discuss data prediction versus data explanation in the design of artificial intelligence — and why we think AI agents should be designed to behave like curious scientists: noumenal.ai/post/why-ai-shou…
Here’s the tl;dr 1/7
Jason Fox retweeted
First blog post by Noumenal Labs on the “DeepSeek moment”: noumenal.ai/post/the-deepsee…
Jason Fox retweeted
Happy new year! Apologies for shitposting, was too funny 😂😂 @mjdramstead
Jason Fox retweeted
🚨 Preprint 🚨 “Active inference and intentional behaviour,” from @VERSESAI (Friston, @TommSalvatori, @a_tschantz, @exilefaker, Verbelen, @KoudahlM, @aswinpaule, @drclbuckley, me) and collaborators (@TakuyaIsomura, Parr, @adeelrazi, @ANeuroExplorer)
arxiv.org/abs/2312.07547 1/8
Jason Fox retweeted
Our new preprint led by the great Karl Friston, in which we explore how the structure of generative models can be learnt from data
arxiv.org/abs/2311.10300
Jason Fox retweeted
Interested in Bayesian Inference and HCI? Open Ph.D. position on Interactive Model-based Probabilistic Visualisations for Exploring Decisions, with @ssteinresearch and myself, linked with the @ERCDIFAI project and @Aegeanairlines. Deadline end of November. difai-project.org/aegeanphd.…
Jason Fox retweeted
Conversation with Dr. Jeff Beck from @versesai on #activeinference the Bayesian Brain, emergence and optimisation. youtube.com/watch?v=c4praCiy…
