I am in SF for a week or so; I would love to meet research friends working on emerging safety challenges.
Book me for coffee here:
calendar.app.google/iTpwFqaU…
There is a mathematical model in political science where seemingly-democratic setups could lead to authoritarian practices, a.k.a. Democratic Backsliding.
Some say it may be the political crisis of our generation.
Here, Fazl's research warns us of what AI design could be capitalized towards that unfortunate path.
At #AIES, the wonderful Keir and Isaac will be presenting our paper.
In this paper, we show how AI systems can make it possible for western democratic regimes to carry out authoritarian practices by undermining how people hold institutions accountable.
have you considered the regime where watermarking might just improve quality
Replying to @OpenAI
Watermarks have limits. They’re often undetectable, especially in short passages. Rewriting or translating text can completely remove the watermark.
That’s why, for now, only approved researchers will have access to our detector, so they can help us evaluate and improve the technology.
We’re treating text watermarking as an ongoing area of research. We’ll keep testing and improving, with feedback from users, developers, policymakers, and researchers.
openai.com/index/eu-text-pro…
i wondered about the same question, too
i miss google translate, @orf_bnw
Readers added context they thought people might want to know
Gemini shows "Flash Extended" because that optional user-selected mode was enabled for deeper reasoning on complex tasks; standard mode is the default and recommended for simple queries like translations.
support.google.com/gemini/answer/…
androidauthority.com/google-gemini-…
This one is for my friends at COLM interested in model safety monitoring
I won’t be at #COLM
Go see our poster if you’re interested in monitoring AI systems: Can an AI learn to hide what it is doing from a monitor that can look inside the model itself?
Our paper studies how “white-box” monitors can be evaded and how to make them much harder to fool!
mimee retweeted
Introducing Contrastive Language Models (CLMs), a System 1 model that connects actions and states!
Agentic coding: with lightweight finetuning, CLM-8B sets a new SOTA on DeepSWE (81.6%) and Terminal Bench 2.1 (87.6%).
CLM-8B vs. Jev: comparable zero-shot performance on computer-use, gaming, and tool-calling tasks, while being up to 9× faster.
We are releasing the CLM-8B checkpoint, its data, and infra today!
Introducing Contrastive Language Model (CLM): an ultra-fast System One Model trained with a contrastive learning objective that connects states and actions.
CLM-8B is pre-trained on internet-scale data and delivers up to 9× faster inference than Jev ⚡ while achieving comparable performance across computer-use, gaming, and tool-calling tasks.
With lightweight fine-tuning, CLM-8B sets a new SOTA on challenging agentic coding benchmarks, such as DeepSWE (81.6%) and Terminal-Bench 2.1 (87.6%). In contrast, Jev fails to serve as an effective verifier for these long-horizon tasks.
We also build an efficient training and serving infra for CLMs by disaggregating states and actions, allowing their embeddings to be cached and reused independently. This substantially reduces inference latency in settings where the state evolves continuously while the action set remains fixed.
Finally, we establish scaling laws for CLMs and show that the test contrastive loss decreases predictably as a power law in training compute, model size, and dataset size.
📄 Blog: contrastive-lm.notion.site
💻 Code: github.com/Contrastive-LM/CL…
🗣️ Discord: discord.gg/5dAQEDJBs
🤗 Data & Models: huggingface.co/Contrastive-L…
More details on CLM’s architecture, data recipe, and scaling laws in the thread below 🧵
mimee retweeted
Sakana AI welcomes Jürgen Schmidhuber as Chief Scientific Advisor.
sakana.ai/schmidhuber/
Sakana AI is incredibly proud to announce that Jürgen Schmidhuber, universally recognized as the father of modern AI, is officially joining Sakana AI as Chief Scientific Advisor.
For nearly four decades, Jürgen has explored how machines can learn to learn. His foundational work in the 1990s drove core advancements in deep learning and established early frameworks for world models. Crucially, his pioneering innovations in meta-learning opened the very path toward recursive self-improvement.
These ideas have already shaped our own research, from the Darwin Gödel Machine to The AI Scientist. Now Jürgen will help guide our newly formed RSI Lab, whose objective is to trigger a compounding cycle of scientific discovery aimed at improving machine intelligence. We are assembling a critical mass of world-class experts in Tokyo to make this a reality.
Welcome, @SchmidhuberAI !
I got awarded my 1st patent
Unlearning Recommendation Models.
🎉 TikTok gave me $800 (minus a bunch of taxes) and the ops team just tried to direct deposit it to the same account from 5 years ago. It failed and they were like oh well we gotta email her. I was not gonna be told otherwise.
Replying to @jackclarkSF
We made a technical-legal critique of the notion, aimed at updating regulators. Lmk if you are interested in taking a look.
@KelseyTuoc also wrote on it theargumentmag.com/p/when-te…
(worth mentioning the og paper aimed to warn when the companies would not talk about scaling harms)
Safety community may be better off unlearning the notion "privacy = absence of breaches"; way too narrow and missed important levers.
For an overview from incentives to techniques in Privacy ML, see our neurips tutorial, brought to you by technical safety researchers @FazlBarez
privacyml.github.io/
more remarks on
* better copyright law for creators
xitter.cf/MimeeXu/status/2095848…
not to be confused with
* better user privacy design for ai
xitter.cf/MimeeXu/status/2097801…
People fatigue from privacy management* and AI products have the unique potential to change that.
Replicating the dark patterns from ad-tech era when people currently love AI is squandering human progress.
* the whole "hidden control"-based policy presumes privacy to be some sort of individual responsibility, when it should be a given.
With AI, we could be in a world where this goes without saying.
Some of you can't imagine the safety people will feel and the human flourishing that can follow from it.
At a high level, this was the full exploit chain.
1. HEIC/HEIF upload
2. ImageMagick decoding
3. Heap overflow on libheif
4. RCE on community.openai.com
5. Critical OpenAI SSO flaw
6. ChatGPT/Codex takeover
7. Connected GitHub access
8. Internal repo PR #1186742
Imagine writing in a paper, "we train our models on synthetic stories about humans only (no AIs)."
1. what do people think it means ? why is ai as a scientific field allowed to do X' and market it as X.
2. most significantly, why do people get sad when models can tell X from X' (when they seed X' as X at eval time)
am i missing something
in some convoluted way we found 1 issue both countries can get behind for ai privacy :|
People fatigue from privacy management* and AI products have the unique potential to change that.
Replicating the dark patterns from ad-tech era when people currently love AI is squandering human progress.
* the whole "hidden control"-based policy presumes privacy to be some sort of individual responsibility, when it should be a given.
With AI, we could be in a world where this goes without saying.
Some of you can't imagine the safety people will feel and the human flourishing that can follow from it.
Privacy policies on almost all frontier model providers are insane and assuming you have ZDR or any meaningful protection, even on enterprise setting is completely false these days!
Here I'm going to name a few very arbitrary and misleading sentences in ToS of frontier labs that most ppl don't know about. This is all assuming you have already *opted-out*, explicitly of data collection. Default for all frontier models is opt in btw!!
1) the 'explicit feedback': your chats are claimed to be 'protected' unless you provide feedback! what is feedback, you ask? it's thumbs up or down to any message in the chat, or even if you pick 'which is better' response on OpenAI GUI. In that case, you have provided explicit feedback and your chat is collected and retained for *over 7 years*!
We NEED more reviewers.
I repeat, we need more reviewers for ML for Systems.
Help shape the future of agents science, chip design, compilers, … and security!
Call for papers
mlforsystems.org/call_for_pa…
Human reviewer sign-up
forms.gle/iebCmjBH1LtvQ9x8A
We need better laws for training signals. My quick notes:
Using “fair share” litigation as the only way to capture data value isn’t in fact sustainable for indie creators.
Licensing NYT content could have had higher performance impact on earlier models than most indie creators’ combined.
1. If the quality were superior, substantial in size/diversity, and used *earlier* in training.
2. if it did not need much processing, and it was related to user needs.
I fathom indie creators can only get like $1-$20 per article now, even if NYT sets the precedent.
Anyhow, /letting/ companies steal and pay off copyright infringement isn’t sustainable for creators.
New law should look like:
• Stringent (big fines) for future data that’s not provenanced.
• Fair share defined with the structure of -
1. participation (consent, when you join, when you can retract/reverse),
2. quality (public definitions, auditable), and
3. potential profits (the only thing copyright arguably looks at now, and it’s a lump sum).




















