Making models go brrr | Engineering @reflection_ai | Occasional PufferLib contributor
Joined March 2024
- Tweets166
- Following116
- Followers615
- Likes197
Pinned Tweet
Excited to finally share our progress in developing a reinforcement learning system to beat Pokémon Red. Our system successfully completes the game using a policy under 10M parameters, PPO, and a few novel techniques.
Blog posted below
Final Fantasy XII's battle system was 10 years ahead of the zeitgeist. It should be remade, but backed by AI agents.
Open weights are free weights. Open weights are free weights. Therefore, with Beam, everyday can be gym day!
Beam is now available across platforms, with open weights under the Apache 2.0 license - in collaboration with an ecosystem of 40+ partners.
We're releasing:
- Weights, tokenizer, and reference implementation
- BF16, FP8 and NVFP4 weights and a DFlash2 drafter
- The open source harness we built and trained Beam on: Mirror CLI
Weights and model card on Hugging Face: huggingface.co/reflection
At @reflection_ai, I've used Beam to make a search and recommendation engine for research infrastructure and even a harness to play video games!
Would love to hear how the community uses Beam. Don't want to post on twitter? Try our Discord discord.gg/xwPxRBT2U
A quick Reflection: I remember sitting down and monitoring day one of the pretraining run. It's AWESOME to see what Beam became from that day.
Introducing Beam: a highly efficient agentic open model with 501B total parameters and 23B active.
- Frontier reasoning efficiency
- Advances the Western open frontier on coding & agentic tasks
- Trained end-to-end from scratch
Full weights release this month.
Learn more about Beam: reflection.ai/beam
I like to tell people that when working in research infra, you should expect all your assumptions to last weeks at best
Presenting my grand unified theory of ML researcher impact: Your impact is directly proportional to how much pain you cause to infra.
Fundamentally, you can only inflict pain upon infra if your approach actually works. And the better your approach works the more pain infra is forced to endure.
So, to give some examples:
- MoE's add a ton of data-dependent computation => pain (shazeer++)
- GDN/KDA are the most complex architecture I've been forced to care about and a very annoying matrix inversion => pain (sonta++)
- Muon is much more annoying than Adam and causes annoying restrictions on parallelism => pain (keller/jeremy++)
- RL scaling forced many researchers to care about LLM inference and RL infra as a category => pain (tworek++)
Even papers like Attention Is All You Need have lead to significant pain! Before transformers were invented everyone was running small jobs and I never needed to think about kv-caches or 6D parallelism.
Tried to turn it into a skill github.com/drubinstein/terry…
Remember to measure first, optimize second.
My current use case is to profile speedruns for a tool I'm working on. Profiling has helped find inefficient navigation subroutines. It's great.
A few years ago, I wanted to find a way to maximize shade during my long runs in the summer. Recently, I decided to experiment with a solution. Presenting Shady Route Finder.
shadyroutefinder.com . Supports 8 cities, can tell you what side of the street to walk on, mobile and even has a sunny route finder mode on desktop. I've tested some routes, but obviously not all. Any feedback would be great.
I dared him to try an ai assisted native rewrite. As expected, 10k sps at best has now become 4M sps. Nice.
대박! A year ago we announced our series A. Today we’re announcing an amazing partnership with Shinsegae. Who knows what’ll come next?
Reflection is partnering with Shinsegae Group to build a 250-megawatt sovereign AI factory for the Republic of Korea.
Open intelligence. Built on trust between allies. Owned by the nations that need it most.
The future of sovereign AI. Read more in the @WSJ.
