
prev continual learning @google , @PrimeIntellect resident, @huggingface evals | opinions my own
San Francisco, CA
Joined September 2023
- Tweets1.4K
- Following1K
- Followers1.5K
- Likes3.6K
Some thoughts about GPT-5.6-Sol after ~30B tokens:
Sol is the most OCD model I’ve used thus far. It very frequently gets one-shotted by random nits in the codebase and writes a bunch of tests to fix it. Even with fast mode, it’s incredibly slow to do this kind of iterative development, especially when builds take really long. This by itself is not a bad thing, but the worst part is that after 2 compactions, it’s chasing the nitpick / useless goals I never told it to accomplish, rather than the main task. This behavior is so bad, I thought I was messing something up and tried codex, pi, and opencode to figure out if it’s a harness issue, but there is no meaningful difference between the three, which leads me to believe this is a model problem.
AI code has this weird delayed release effect. You’ll only notice slop code 2 dev cycles into a codebase when you spend more time fighting with the code and on refactors than on shipping features. It’s possible that sol is better than 5.5 a couple cycles in, but tbd.
My file deletion experience has also been similar to others: this is a dangerous model to let loose without guardrails. For instance, when performing a routine container upgrade, it accidentally printed out an env secret, then panicked and rotated ALL secrets (this is internal so not public facing, which was also documented), and proceeded to break everything, spending an extra hour fixing everything and redeploying everything else to use the new secrets. It also gets rid of files it doesn’t like. I have no idea why this is, but I think something about the reward model rewarded bookkeeping.
Writing is another problem. 5.6 has a huge context bleed effect. It does not know how to write documentation and starts putting the specs in the documentation. If I ask it to develop a user sandbox for isolation, and also ask it to write documentation, it starts talking about specs and sandboxes in user-facing docs, which makes no sense. Fable is somehow much, much smarter in this regard. Frontend design has also not gotten better. Fable is still one generation ahead here.
Overall, as a huge 5.5 user, I am not convinced that sol is a meaningful upgrade. It’s possible my practices need to change, but unfortunately it feels like I’m spending longer fighting with 5.6 than I did with 5.5. It’s like the model is so SO smart, but so hard to work with, compared to fable and even grok4.5 surprisingly. It’s clearly intelligent, but also just doesn’t care about what I ask it to do? (Is this supposed to be AGI feels like?) I hope the codex team fixes what possibly is a bad harness setup, because the benchmark numbers show a very different story from what I’m seeing while using the model.
this is a large part of the reason i churned from oai as someone who used to use multiple $200 accts.
you think claude is bad, but gpt is worse. it won't do basic shit like even clicking yes on a form, EVEN IF I AUTHORIZE
it then gives me a billion useless caveats "one thing to point out" like FUCK OFF
this was a large part behind me daily driving abliterated deepseek for a while, because it just felt like i had my big boy pants on instead of playing in the kiddie pool with floaters
@thsottiaux pls fix so i can resub
Claude can't see Reddit, b/c Reddit blocks all LLMs except Gemini, b/c they have a Google contract. I ask Gemini for posts matching criteria, but Google blocks direct quotes. So I ask Claude to use a headless browser, which it refuses for 'safety'. Fuck everything about this.
2
8
675
this is UNIRONICALLY what is going to happen
>check in on European "sovereign AI"
>comparing against 6-month-old models on 2-year-old benchmarks
>ask the Swiss dev if it's actually built from scratch or just Chinese
>he laughs and says "it's a larger exercise sir"
>look inside
>it's Kimi
Today we announce the launch of the Giotto family of models. We are unveiling Giotto Alto and Giotto Solo, the smartest European sovereign AI models on the market.
Giotto Alto is our flagship model built for advanced reasoning capabilities, free for everyone and available on the App Store and Google Play. It is hosted on European cloud infrastructure, so your data stays in Europe. Giotto Solo is the smartest model that runs on a single GPU, so organisations can keep their AI on their own infrastructure.
We are a Swiss lab with a European heart and we built this so that Europe can run advanced AI on its own terms.
Use Giotto: cloud.giotto.ai/
Readers added context they thought people might want to know
Giotto's graphic shows self-reported benchmark scores comparing to specific variants of other models; independent leaderboards as of late Oct 2026 list no Giotto and different leading scores on AIME 2026, GPQA-Diamond and HLE.
giotto.ai
benchlm.ai/benchmarks/aim…
benchlm.ai/benchmarks/gpq…
benchlm.ai/benchmarks/hle
4
35
923
Interesting foundation models for geospatial imagery with a new set of inputs compared to traditional text, vision, audio approach
first principles reasoning makes you wonder why you want to fit a square peg into a round hole for hyper specific tasks with different data shapes
Meet Finch 1, our first step toward one model that works with any sensor, at any resolution, anywhere on Earth.
Trained on imagery from 50+ sensors across satellites, aircraft and drones.
5
333
shuhaib is incredible, and I think a representation for user behaviors is going to prove to be super useful as we scale up RL
this is your sign to short accenture
leadership that is completely clueless that an engineer w/o claude is nearly useless compared to an engineer w/ unlimited claude
New: Accenture has told employees it will only fund $29 per month of Claude use. Meaning that employees who often spend as much as $1,000 a month are now scrambling to find ways to keep access to the tool. A sign of the continuing crack down of AI 404media.co/accenture-tells-…
2
18
1,201
Sumuk retweeted
🧵[1/N] Are large language models more creative than humans at combining ideas? 🤔
Combinatorial creativity involves combining familiar ideas in unfamiliar ways, and is central to scientific discovery and technological innovation.
In fact, breakthrough scientific work is often associated with more distant combinations of methods, techniques, and concepts.
Here, we introduce Kombine, a new benchmark of combinatorial creativity for LLMs.
TLDR:
- LLMs are more combinatorially creative than the average human.
- However, conceptual blending is the hardest task for LLMs, and even frontier models struggle.
- LLMs also tend to homogenize, inventing concepts with the same properties across separate models.
- Moreover, humans and LLMs combine in distinct regions of semantic space.
- As a result, we anticipate greater conceptual diversity can be unlocked through human-AI collaboration. 🗣️
Thread below🧵⬇️
this is insanely tone deaf wow
This is a slide from Terence Tao's 'Math 2.0' public lecture yesterday evening at Caltech. I am highly confident that, if the cure actually worked, almost any Stage 4 cancer patient would tell you that human comprehension of its mechanism is completely irrelevant to them.
2
12
819
honestly starting to hate terry tao these days. he clearly has never met someone actually willing to do ANYTHING for a chance at survival.
these egomaniacal clowns will try stopping a cure for cancer if it doesn’t help their own status that they’re the only ones to understand it.
if I’m dying from cancer with 3 months to live, I don’t give a shit if a human mathematician understands it. That should NOT be a blocker for progress
o wow this must be a tiny fucking model if they’re modeling in batch density into the public pricing scheme
Replying to @claudeai
Haiku 5.5 is especially good value on tasks under 100,000 tokens, which make up around 90% of requests to our previous Haiku model.
2
247
this is such a sad state of affairs, especially with the mob hating @thsottiaux within a week of being in love with him. yes OpenAI is lowering the juice values, but what do you expect them to do?
> if they don’t, people complain that it is too slow, usage limits are bad, etc
> if they do, people complain it’s quanted
the fundamental problem here is that tokens are a usage based utility being sold as a subscription. this is like early energy companies selling electricity on a monthly plan.
funnily enough, the subscription model breaking is evidence that the product itself is working extremely well.
I understand the mechanics and need to do this, but this is where it essentially ends up leading to misaligned incentives. it’s like the power company fucking with your amperage at times or some of your electricity failing to run your dryer randomly
my long term bet is that everything will have to move to usage based tiering, ONCE capable models are cheap enough. I would estimate this to be around gpt-6.1-sol capability in Luna pricing, with off peak discounts, etc. then the general public may be able to stomach.
either that or local models take over like what @iamgingertrash believes (having solar panels on your roof) and you call frontier models (tapping into the grid) when you need orchestration or some larger capability and pay for it on demand
🚨 Same GPT-6, less thinking in Codex : lower juice than the API at every effort level, up to 12× less on max 🧐
Astra on max 832 on api vs 128 in codex , Sol on max : 768 juice on API vs 62 in Codex
1
3
457
“Why are they trying to solve these problems? They should let me make a career by being a meat proxy to solve it instead” is a CRAZY hill to die on
The scale of this is staggering.
But... Why?
Why is OpenAI trying to solve hundreds of math problems to be released all at once? I think we got the point when they found a counterexample to Navier-Stokes -- internal model = great.
I am not sure at this point what this (hostile?) takeover of the mathematical landscape is trying to achieve other than, again, a massive PR stunt.
Sumuk retweeted
final week to apply to Speedrun Alpha fellowship! up to $250k investment in technical students + recent grads who want to build. apps close this sunday. 👀 👀 👀
apply: alpha.a16z.com
details below 👇
–
alpha is our fellowship for technical students and recent grads who want to start building. we have two tracks:
for future founders:
- up to $250K investment
- $1M+ in credits across AWS, GCP, Azure, OpenAI, Anthropic, Cursor and more
- office hours and fireside chats with speedrun partners
- 8 weeks in person in SF (jan 11 – mar 5)
if you'd rather get experience before starting a company, apply to our talent track:
- intros to speedrun portfolio companies to find your mutual fit
- office hours and fireside chats with speedrun partners
- 4 weeks in person in SF (feb 8 – mar 5)
david is one of my favorite people to follow. like this benchmark is so trivial yet so important in ways that others really haven't explored
i asked LLMs to have fun, write down what happened, then rate who had the most fun. the newer models seem to have less fun! many models played word games, but opus 4.6 won over and over by imagining brief worlds made up of contradictions living inside falling water droplets 🥺
4
562
damn @alexandr_wang living rent free in their heads
openai cfo: muse is going to better, sorry, not muse, dots is going...
youtube.com/watch?v=jXvxK87F…
4
524






























