Brian T. Kent

AI just learned to fine-tune itself between questions. MIT introduces SEAL, a framework enabling LLMs to self-edit and update their weights via reinforcement learning, all by itself. LLMs consume whatever data they are given, so they stay frozen after pretraining. SEAL teaches a model to write its own study material, fine-tune on it, and keep learning. ⚙️ The Core Concepts The core idea behind SEAL is to enable language models to improve themselves when encountering new data by generating their own synthetic data and optimizing their parameters through self-editing. The model’s training objective is to directly generate these self-edits (SEs) using data provided within the model’s context. Each self-edit is a text directive that specifies how to make synthetic data and set hyperparameters for updating weights. The generation of these self-edits is learned through reinforcement learning. The model is rewarded when the generated self-edits, once applied, lead to improved performance on the target task.

Brillaint 🔥 @GoogleDeepMind launched AlphaEvolve, a Gemini-powered coding agent that discovers and evolves algorithms, outperforming its predecessor AlphaTensor. It achieved 23% matrix kernel speedup, 32.5% GPU kernel boost, and 0.7% compute recovery in data centers, driving major efficiency gains in AI training, chip design, and math problem-solving. ⚙️ The Details → AlphaEvolve uses an ensemble of Gemini models—Flash for exploration and Pro for deep refinement—to generate and evolve code for algorithmic tasks. Programs are scored via automated evaluators, enabling objective progress tracking. → It discovered faster matrix multiplication algorithms, including a 4x4 complex matrix multiplication using only 48 scalar ops, improving on Strassen’s algorithm. → It recovered 0.7% of compute resources globally via improved data center scheduling with interpretable heuristics. → Contributed Verilog-level enhancements to Google's TPU design pipeline. → Delivered a 23% speedup in matrix ops and 1% reduction in Gemini model training time. → Achieved 32.5% acceleration on FlashAttention GPU kernels, reducing dev cycles from weeks to days. → AlphaEvolve doesn't just optimize—it innovates. Given only a minimal code skeleton, it co-designed a novel gradient-based optimization procedure, creating multiple new matrix multiplication algorithms. → Solved parts of 50+ open math problems, improving 20% of them, including the 11D kissing number problem with 593 spheres. → It rediscovered state-of-the-art solutions in 75% of experiments across number theory, geometry, analysis, and combinatorics, validating its mathematical depth. → In 20% of cases, AlphaEvolve pushed the frontier, like improving the kissing number problem in 11D with a new configuration of 593 spheres, setting a new lower bound. → Setup time for most math problems was just hours, enabling fast iteration and exploration across diverse domains. → Early access for academics is being planned, signaling potential broader availability.

Nice collection of LLM papers, blogs, and projects, focussing on OpenAI o1 and reasoning techniques. What it offers: 📌 Curates papers, blogs, talks, and Twitter discussions about OpenAI's o1 and LLM reasoning 📌 Tracks frontier developments in LLM reasoning capabilities and techniques