Mithil Vakde (@evilmathkid) header
Mithil Vakde (@evilmathkid) profile picture

Trying to solve sample efficiency prev: Engineering physics @iitbombay '23

Joined June 2023
Photo by Mithil Vakde (@evilmathkid)Photo by Mithil Vakde (@evilmathkid)Photo by Mithil Vakde (@evilmathkid)Photo by Mithil Vakde (@evilmathkid)Photo by Mithil Vakde (@evilmathkid)Photo by Mithil Vakde (@evilmathkid)Photo by Mithil Vakde (@evilmathkid)Photo by Mithil Vakde (@evilmathkid)Photo by Mithil Vakde (@evilmathkid)
Announcing ⚡⚡ 1) New records on ARC-AGI public eval !! 2) A new sample-efficiency benchmark 70% on ARC-1 with just 19M params (15% on ARC-2) 49% with just 1M params!! (Latter beats TRM's 45% with ~400x fewer params) Trained from scratch on only 1,307 ARC puzzles. Just ~$13 to train No text, no LLMs 🧵
23
24
4
184
16,059
Middle layer looping! This is what helped my ARC model score incredibly high Try out ideas like this on ARC Data Squeeze to improve sample efficiency!
Some of you thought that Inkling 2 Small was too big at 594B parameters. We heard you: Inkling 2 Mini is a looped transformer with just 26B parameters, small enough to run on your laptop, powerful enough for agentic work with a score of 20.1 on AA Index. thinkingmachines.ai/news/ink…
2
1
47
1,574
Announcing ⚡⚡ 1) New records on ARC-AGI public eval !! 2) A new sample-efficiency benchmark 70% on ARC-1 with just 19M params (15% on ARC-2) 49% with just 1M params!! (Latter beats TRM's 45% with ~400x fewer params) Trained from scratch on only 1,307 ARC puzzles. Just ~$13 to train No text, no LLMs 🧵
1
130
People have already built on top of mdlARC! @kschweig_ 's transductARC got 76% on ARC-1 with extra data (RE-ARC), scale aug and answer filters With the same data and tricks, looped mdlARC gets 81% (20% on ARC-2) These break the rules, so they go on a separate Open leaderboard teapawt.net/kschweig_/status/20994…
76% on ARC-AGI 1 training from scratch + inference in a bit more than 4h on a single H100. Throughput-optimized recipe attains 44% (TRM-level performance) in 17min. Just a simple autoregressive transformer with a few twists. 🧵
2
4
401
44% on ARC-AGI-1 in 67 cents! Trained from scratch in 2hrs on a 5090 Matches TRM, beats HRM and is way faster & cheaper No recursion, just a transformer Also, 7% on ARC-2 🧵
1
355
Mithil Vakde retweeted
Scaling in almost every domain is bottlenecked by data. That makes data efficiency the most important algorithmic problem in AI. I'm thrilled to share *universal* data-efficient algorithms with massive gains, up to 56x data efficiency for molecules, 24x for DNA, and 24x for crystals against SOTA autoregressive recipes. After fine-tuning, we beat models pretrained on hundreds of times more data. I believe this will fundamentally change our approach to data-constrained scaling across all domains. w/ @asatheesh6561, @bishmdl76, @madhavsinghal_, @akshayvegesna
20
46
12
360
22,251
Mithil Vakde retweeted
we should not like a future where there are 1 to 2 AIs and the rest are illegal. we should want a cambrian explosion of the types and possibilities of minds, experiencing new joys and thinking new thoughts. ensuring that this is compatible with human flourishing is only step 1
179
363
47
3,656
132,778
Mithil Vakde retweeted
LLMs require very large amounts of data and arguably the path to ASI runs through a different model architecture that’s more data sparse. Amazing what @evilmathkid has done with compute constraints on ARC-AGI-2. cc @fchollet
I got 37% on ARC-AGI-2 Kaggle without LLMs! 100M looped transformer trained from scratch - Likely highest score without an LLM - Peaked at 6th place - Beats last year's winning LLM - Looping, added in the last week: 33% → 35% → 37% thanks to @arcprize @fchollet @mikeknoop and @GregKamradt for the competition! Link: mvakde.github.io/blog/looped… (writeup for the paper track)
2
11
803
Mithil Vakde retweeted
And Mithil strikes again, this time casually beating opus
I got 37% on ARC-AGI-2 Kaggle without LLMs! 100M looped transformer trained from scratch - Likely highest score without an LLM - Peaked at 6th place - Beats last year's winning LLM - Looping, added in the last week: 33% → 35% → 37% thanks to @arcprize @fchollet @mikeknoop and @GregKamradt for the competition! Link: mvakde.github.io/blog/looped… (writeup for the paper track)
2
1
13
763
THE GOAT (of resource efficient arc impls) RETURNS
I got 37% on ARC-AGI-2 Kaggle without LLMs! 100M looped transformer trained from scratch - Likely highest score without an LLM - Peaked at 6th place - Beats last year's winning LLM - Looping, added in the last week: 33% → 35% → 37% thanks to @arcprize @fchollet @mikeknoop and @GregKamradt for the competition! Link: mvakde.github.io/blog/looped… (writeup for the paper track)
1
8
549
I got 37% on ARC-AGI-2 Kaggle without LLMs! 100M looped transformer trained from scratch - Likely highest score without an LLM - Peaked at 6th place - Beats last year's winning LLM - Looping, added in the last week: 33% → 35% → 37% thanks to @arcprize @fchollet @mikeknoop and @GregKamradt for the competition! Link: mvakde.github.io/blog/looped… (writeup for the paper track)
52
37
7
540
25,176
Why do it without an LLM? Because I care about sample efficiency. LLMs see so much data, ARC puzzles included, that you can't measure it. We now know that with enough data, ARC-1/2 become trivial So the interesting question is how far a model gets on a few ARC puzzles alone
1
4
932
This is a scaled up and upgraded version of my previous work on ARC-AGI teapawt.net/evilmathkid/status/202…
44% on ARC-AGI-1 in 67 cents! Trained from scratch in 2hrs on a 5090 Matches TRM, beats HRM and is way faster & cheaper No recursion, just a transformer Also, 7% on ARC-2 🧵
1
6
910