James Harrison (@jmes_harrison) profile picture

Cyberneticist @GoogleDeepMind

San Francisco
Joined September 2014
Photo by James Harrison (@jmes_harrison)Photo by James Harrison (@jmes_harrison)Photo by James Harrison (@jmes_harrison)Photo by James Harrison (@jmes_harrison)Photo by James Harrison (@jmes_harrison)Photo by James Harrison (@jmes_harrison)Photo by James Harrison (@jmes_harrison)Photo by James Harrison (@jmes_harrison)Photo by James Harrison (@jmes_harrison)Photo by James Harrison (@jmes_harrison)
James Harrison retweeted
Deepseek R1 inference in pure JAX! Currently on TPU, with GPU and distilled models in-progress. Features MLA-style attention, expert/tensor parallelism & int8 quantization. Contributions welcome!
8
47
6
290
48,402
Are you still using hand-designed optimizers? Tomorrow morning, I’ll explain how we can meta-train learned optimizers that generalize to large unseen tasks! Don't miss my talk at OPT-2024, Sun 15 Dec 11:15-11:30 a.m. PST, West Ballroom A! teapawt.net/benjamintherien/status…
Learned optimizers can’t generalize to large unseen tasks…. Until now! Excited to present μLO: Compute-Efficient Meta-Generalization of Learned Optimizers! Don’t miss my talk about it next Sunday at the OPT2024 Neurips Workshop :) 🧵arxiv.org/abs/2406.00153 1/N
6
33
2,328
New: a neural net-based approach to Bayesian optimization that performs well in both classic, small-scale problems, and can efficiently scale far beyond GP surrogate models. If you're at NeurIPS, come by our poster at the Bayesian decision-making workshop today! More info👇
3
1
26
2,173
Our results show our approach of VBLLs + MLP features + our training approach yields SOTA or near-SOTA performance on a bunch of problems from classic 2D Ackley, all the way to much more challenging problems like Lunar Lander controller tuning
1
2
211
Our approach can be applied to a very very wide range of problems---we include multi-objective BayesOpt as an example, but we're excited about massively scaling up this approach on a ton of problems. Paper link here: arxiv.org/pdf/2412.09477
1
169
Going to @itssieee ITSC'24? Check our tutorial on Data-driven Methods for Network-level Coordination of AMoD Systems Organized with @DanieleGammelli, Luigi Tresca, Carolin Schmidt, @jmes_harrison, Filipe Rodrigues, Maximilian Schiffer, @drmapavone rl4amod-itsc24.github.io
2
4
11
1,645
Want a really simple (and cheap!) way to improve neural net calibration and get practical epistemic uncertainty estimates? At ICLR this year: Variational Bayesian Last Layers Try it out: 1. pip install vbll 2. a couple one line changes to your current training pipeline 🧵
3
24
2
122
15,529
We tested VBLLs in regression, classification, and active decision-making. I'm particularly excited about the bandit performance---VBLLs enable nearly free active learning! Plus, you can use VBLLs jointly with other Bayesian methods, or add them to your model post-training.
1
4
807
A question we have been thinking about for a long time: what is the natural architecture for a learned optimizer? We now have an important part of the answer---we can automatically construct expressive optimizers based on optimizee network symmetries. Check out Allan's thread!
🧵: How do you design a network that can optimize (edit, transform, ...) the weights of another neural network? Our latest answer to that question: *Universal* Neural Functionals (UNFs) that can process the weights of *any* deep architecture.
2
37
12,247
James Harrison retweeted
📝Quiz time: when you have an unrolled computation graph (see figure below), how would you compute the unrolling parameters' gradients? If your answer only contains Backprop, now it’s time to add a new method to your gradient estimation toolbox!
1
13
1
128
35,566
Looking forward to getting started at #ICML! Happy to chat about RL, learning-based control, and Graph ML. Make sure to drop by our poster! (Wed 26 Jul 2 p.m. PDT)
Excited to share that our paper on Graph-Reinforcement Learning was accepted at #ICML2023! We present a broadly applicable approach to solve graph-structured MDPs through the combination of RL and classical optimization. Website: sites.google.com/stanford.ed… 🧵👇(1/n)+quoted tweet
12
59
24,526
Graph deep learning and bi-level RL seem to work exceptionally well for a whole bunch of critically important real-world problems like supply chain control. Plus, it easily combines with standard linear programming planners in OR. Check out @DanieleGammelli's thread for info!
Excited to share that our paper on Graph-Reinforcement Learning was accepted at #ICML2023! We present a broadly applicable approach to solve graph-structured MDPs through the combination of RL and classical optimization. Website: sites.google.com/stanford.ed… 🧵👇(1/n)+quoted tweet
1
4
757
Really nice + concise VeLO explainer!
Why tune optimizers hyperparameters (ex. Adam) by hand, if one can train a neural network to behave like an optimizer and dynamically find the best update for your neural network’s weights? In this video, we explain the VeLO learned optimizer!👇 📺 youtu.be/9a6PQJxzUpM
1
12
1,658