covering AI news, working on @voxelbench
Joined October 2018
- Tweets3.5K
- Following956
- Followers15.3K
- Likes43.7K
Legit retweeted
Models can seemingly degrade over time.
The following scenes were created with the same prompt and the same model, @GoogleAI's Gemini 3.8 Flash, by a user on DesignArena.
The first was generated just 3 days after release, while the second came almost a month later.
The difference is striking. The first scene features a finished bungalow with terraced gardens, while the second is barely recognizable.
Digging into the code, we find that the second scene still attempts to build the garden, windows, and patio, but buries much of it beneath its own terraces.
In aggregate, the share of objects never visible on camera jumps from 18% to 57%. For bushes, the share placed underground jumps from 0% to 100%, and even the house walls go from fully above ground to 64% below it.
Both scenes share the same simple user request, yet a few weeks after release, the model struggles to fulfill it. Try making your own generations and judge for yourself at DesignArena.ai!
Legit retweeted
Kimi K3.1 Pro has ranked 6th on the VoxelBench Live Leaderboard!
currently tied with Gemini 4 Argon
Legit retweeted
🚨🚨ALERT: ANTHROPIC IS A/B TESTING USAGE LIMITS
Many users on X are reporting LOWER usage limits right now. THEY ARE NOT WRONG. We detected an A/B test going on that DECREASES your monthly API-value by ~20% while INCREASING your 5h usage limits by ~20%. This only affects SOME accounts. (1/2)🧵
Argon is out on VoxelBench
its performance is very uneven; sometimes producing clean and detailed builds, while also often making much less detailed and smaller builds
overall im underwhelmed, but I hope they recognize this gap and improve it fast
one similarity I found interesting (and annoying) between this model and Fable 5.0 is that they both prefer to make smaller builds
but unlike Fable, Argon made no effort to verify that the blocks it placed weren't overlapping or clipping into other blocks
on average, Argon spent 2 minutes per VoxelBench prompt; far below the average of models like Opus and Fable, and even Astra or Sol
Legit retweeted
Business Insider is reporting that Google is internally testing a new Gemini 4 checkpoint named Carbon that matches Opus 5.5 in coding.
Legit retweeted
We’re releasing a broad range of new mathematical results produced by an internal frontier model.
We’ve been consulting with the independent Advisory Group on Mathematics and Artificial Intelligence at the Institute for Advanced Study, and we have drawn on their advice and public recommendations to inform how we release these results.
github.com/openai/math
Legit retweeted
anthropic is testing haiku 5.5 too
been using fable 5.5 for the last few days but noticed something weird
some outputs from fable 5.1 that knows tibo the reset guy were way worse than sonnet 5.5
but why would it give worse outputs?
so i can only think of one thing that team is routing some prompts to haiku 5.5 through fable 5.1 too and testing it that way
which makes me think we might see haiku 5.5 in next week or two
here's one weird output i got
just look at how different these outputs are even tho both were run on fable 5.1 xhigh that knows tibo the reset guy
Legit retweeted
Apparently, the same OpenAI models may get way more reasoning budget through the API than through Codex.
So even if you’re using OpenAI’s own Codex harness, you could still be getting a more constrained version of the model compared to someone just paying for the API directly.
For example, GPT-6 Astra is reportedly around 832 “juice” on Max via API, vs 128 through Codex.
(If this is true, and the measurements suggest it might be, then OpenAI is basically holding these models back on its own plans while API users get the real monsters.)
