notebook of future PhD research directions and unrelated hot takes
NY | SF
Joined February 2022
- Tweets745
- Following708
- Followers536
- Likes3K
not that we have to do it like the brain...but there's just no way the brain is this sensitive to numerical noise
it's just a remarkably bad experience
- fixed precision arithmetic is not associative -> can't reliably accumulate gradients across machines; order of operations matters
- optimizer state, RNGs, dropout, recomputation all going in the background
brian kitano retweeted
1/ I was pretty excited when I saw this yesterday. I thought it would be interesting to see if GPT-6 Astra could edit the weights of the 500M-parameter Celeste world model I trained recently and improve how well the simulation holds up when playing in it. It turns out it could!
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math.
It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training.
We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵
Blog: rikiyat.github.io/blog/weigh…
Readers added context they thought people might want to know
LLM was given weights explicitly as text in prompt and used code/forward passes (per authors' logs) to ablate Qwen looping direction, a known thinking-mode issue per model card. Primarily fixed repetition in that mode, not general 'natively' improving weights/math.
rikiyat.github.io/blog/weight-sp…
github.com/RikiyaT/lm_und…
huggingface.co/Qwen/Qwen3.5-0…
huggingface.co/RikiyaT/Qwen3.…
you can make an environment where SFT on passing traces does not improve performance...now time to figure out what to change and retry
wow - long horizon SFT is so memory hungry.
each token: 5120d @ FP16 (2 bytes)=10kB. at 100k context, that's 1GB per sequence. at 64 layers, that's ~64GB of activations stored in memory.
H200 only has ~140GB, so we have to use context parallelism for larger models or batch size.
benchmarks are fundamentally economic in nature - not just "economically valuable" but "inspired by economics". ELO, market competition, etc is how to drive relative performance forward and identify capability gaps intersubjectively, rather than objectively.
brian kitano retweeted
Intelligence must draw on memory. The mind must not simply compile data — as an algorithm now does more quickly than we can — but recall lived experiences, which contain depths of meaning that only the human soul can recognize, and connect more deeply with the goodness of these experiences. Everything that is good begins “with the Spirit” and thus possesses a breath, a dynamism and a sense of communion that, once experienced, are engraved upon the heart. vatican.va/content/leo-xiv/e…
brian kitano retweeted
1/ I've been working on inverse dynamics models and gameplay policies recently (blog post soon) and wanted to see how well a Jev-style System One model could do in the control loop. I self-hosted Kev-8B and wired it into Celeste, the primary game I've been training models on.
"faster horse" another way of saying "latent pareto axis" which makes the jevon of it all more salient
corollary: make everything search. what does this mean?
1. make hot lava problems not hot lava problems - the easiest way to do this is simulation (then there's no real lava to step on)
2. (short term) add more affordances to make the Landscape less Zhangjiajie - denser reward functions, etc.
Is everything just search?
- math
- games
- gradient descent
No, some things are not search:
- no approximate manifolds: jumping from island to island / floor is lava
- frictional manifolds: there's cost to exploring that outweigh reward from arriving
There's also a time constraint (patients will die, proofs take 80 years, etc), so we train actors (people, agents) to have search heuristics to find solutions faster.
Heuristics modeling often incorporates causal modeling (biology/chemistry -> medicine). Math is special because "causality" is deduction, whereas much else is inductive/abductive.