Sebastian Schmon (@SeBayesian) header
Sebastian Schmon (@SeBayesian) profile picture

Machine Learning @Latent_Labs, anything from classical statistics to modern machine learning and generative AI, previous @altos_labs

United Kingdom
Joined June 2020
Photo by Sebastian Schmon (@SeBayesian)Photo by Sebastian Schmon (@SeBayesian)Photo by Sebastian Schmon (@SeBayesian)Photo by Sebastian Schmon (@SeBayesian)Photo by Sebastian Schmon (@SeBayesian)Photo by Sebastian Schmon (@SeBayesian)Photo by Sebastian Schmon (@SeBayesian)Photo by Sebastian Schmon (@SeBayesian)Photo by Sebastian Schmon (@SeBayesian)
Saying there is a "distribution shift" between train and test data is a great marketing ploy in #machinelearning to make overfitting somehow the data's fault, not the model's or modellers'. 🙃
7
9
1
71
No, the issue is that the whole idea of UBI and AI-produces-everything assumes a static world. Humans aren’t static, they won’t be content in a world with monthly paychecks and nothing to aspire to. It’s naive to conceptualize the world as a “condition” and not “evolution of conditions”
Looking at the comments, it seems people can’t conceptualize that if we produce the same amount with no human labor, our material conditions would be better, not worse
3
1
6
479
This weirdly assumes that people would simply accept such a society?
Anthropic Researcher Tao Lin: Pretty much all humans on earth will become permanently unemployable because of AI "It's very likely that pretty much all humans on earth will become permanently unemployable" "And will need to strictly rely on government welfare to sustain themselves" "Human employment will go away."
3
148
One upside of LLMs is that we are no longer discussing removing page limits in ML papers.
1
1
9
260
Now back to reviewing papers with 100(!!!) pages of appendix…
24
Sebastian Schmon retweeted
We’re excited to announce DiG-bench, a new benchmark for discovery! Over the last few weeks we’ve been testing frontier AI models on our novel discovery games and seeing how they score. Each game is a text-based environment, so they probe discovery capabilities in the natural domain of language models, rather than requiring additional, potentially confounding, visual understanding. TL;DR frontier models have improved a lot over the last few months. But they are still stumped by some surprisingly simple problems, even in their native text domain. With @cocosci_lab (@Princeton) @MITCoCoSci (@MIT) @SchmidhuberAI (@KAUST_News) @misovalko (@Inria) @tri_dao (@PrincetonCS) @RMBattleday @zebkDotCom @FraserGreenlee @akaijsa @ClareMaguire @TimMuller1 @kubicek_ales @physicscat0x7d @SukritSumant @thoughtchannel_ (1/5)
44
325
242
1,181
1,198,652
Sebastian Schmon retweeted
Today we are happy to share one of our first open research releases: Latent Sparse Attention (LSA), a step towards subquadratic attention that matches accuracy of full attention at ultra-long context. Sparse attention attends to only a few tokens per query, but it has to find the right ones. At 10M tokens, that search takes 98% of the attention layer's time. LSA makes sparse attention up to 22.7× faster and full attention 72.4x faster. With @kubicek_ales @caciolai @PengLiangzu @SukritSumant @Ben_Hoov @FraserGreenlee @RMBattleday @radim_urban @thoefler @jcrwhittington
100
338
82
2,845
509,367
PSA: everyone can tell if your design came from Claude
1
143
I cannot get over the fact that Max is not the maximum.
We’re renaming our Max plans: Max 5x → Max Max 20x → Ultra Max is still $100/month, and Ultra is still $200. If you’re a subscriber, there’s nothing you need to do. We’re also removing the 5-hour limit for verified Ultra accounts, starting today with most existing subscribers. Newer accounts keep a temporary limit until they’re verified, which helps us prevent fraud and abuse. You'll get an email when it's removed. Weekly limits remain unchanged. View our plans here: claude.com/pricing
1
3
124
The extent to which you can just do things now is staggering: I used AI to generate my own RTS, including the possibility to play against an agent by creating a machine readable gaming engine. I am still wrapping my head around how this will change industries.
1
5
187
I used @claudeai for orchestration, @bfl_ai for designs drafts, @ChatGPT for blender models, @ElevenLabs for voices and sounds. Implemented using @godotengine
2
60
One thing that @OpenAI could do with the massive compute is to brute force re-review every research papers, beginning with maths and physics and continue to stats and ML.
4
204
Unpopular(?) opinion: we don’t need classifiers to detect whether something is LLM generated or not, but rather whether it’s worth reading, i.e. we need to detect slop vs non-slop, not AI vs human. Those are very correlated but not identical.
4
10
245
Claude has written me some of the best explanations on topics that were absolutely worth reading and there is a lot of human generated text on the internet that is factually wrong or at least badly written.
52
In the era of LLMs conciseness is the proof of thought.
omg I love the term "slop grenade" it's perfect thx @tobi
2
200
Sebastian Schmon retweeted
omg I love the term "slop grenade" it's perfect thx @tobi
70
133
61
2,644
242,734
I miss the old days where Cursor would just write a for loop for me and then wait for me…
3
236
Is there a word for a mistake that almost every paper in a field makes, that “everyone knows” is a problem, so nobody can publish the correction? (Rejected as trivial because well known)
4
223
Sebastian Schmon retweeted
uatx.substack.com/p/forget-y… Hmm. I'd say the Enlightenment version of individualism is the historically unique idea that individuals exist not for the sake of their ruler, their class, their ancestors, their gods, or any other such superstition, but for their own sake. Meaning does not require subordination to something "greater than oneself". It is created by the idea that progress is possible and desirable – not by a leap to utopia, but endlessly into the future by undirected problem-solving, carried forward voluntarily by individuals for their own reasons.
35
120
27
644
70,650