Digital transformer
Milan
Joined March 2009
- Tweets56.5K
- Following554
- Followers776
- Likes1.7K
Pinned Tweet
Beyond De-Skilling: Intelligence Explosion and the End of Skill as a Stable Category
4m4.it/posts/beyond-de-skill…
Antonio Montano ☼ retweeted
An agent can stop rereading a source without making that source disappear from the product.
Tsinghua’s Coca-Cola paper compresses tool output into continuous vectors instead of a shorter written summary. The compressor is trained to preserve the agent’s next action.
The authors report 70–89% lower total agent token use and 24–56% shorter runtime across four web-search QA benchmarks, with higher answer scores than the uncompressed baseline.
The product implication I find interesting is evidence retention.
Imagine a research agent comparing suppliers. It may not need every paragraph in its working context, but I still need to open the passage behind “delivery takes two weeks.”
Keep the source text, retrieval time and relevant excerpt accessible outside the model context, with the recommendation linked back to them.
The study tests bounded web QA, not this product design. Still, it gives builders a reason to design what the model consumes separately from what a person can inspect.
arxiv.org/abs/2611.06500
Antonio Montano ☼ retweeted
The Alignment Tax Is Graded: Task-Selective Capability Loss Under Length-Normalized DPO with SFT Retention
Ye Yuan
arxiv.org/abs/2611.06476 [𝚌𝚜.𝙻𝙶]
💬Code: github.com/yuan-phd/delta-fi…
ALT Setting. This is an empirical characterization study; we propose no new method and run no comparisons against alternative objectives. We fine-tune Qwen3.5-4B with sparse LoRA and length-normalized DPO plus an SFT retention term (the RPO objective) on structured financial-filing tasks, and track downstream capability as the implicit reward margin grows to ∼ 7 β-scaled units (nf4 training; ∼ 3 under fp16 measurement on the unquantized base). Question. Does margin growth degrade capability uniformly across tasks, or selectively? Findings. In a pre-registered study (8 seeds, 6 unseen at freeze) with artifact-aware scoring and TOST equivalence testing, the alignment tax is graded and task-selective: extraction is preserved (Δ=-0.55 pts, equivalent at ±5 pts), json loses little (Δ=-3.52 pts, equivalence fails on a clustered item-level interval), and router degrades severely (Δ=-30 pts). Router's loss concentrates in a margin 1-3 damage window containing a transient format-break spike. The ma
Antonio Montano ☼ retweeted
AI agents have completely changed my personal theory of economic value.
Now, anyone can become technical—and every knowledge-worker is a product engineer.
choosingvictory.com/p/the-ri…
Antonio Montano ☼ retweeted
“The Gold Rush in AI4Math, II: What are signatures of AI-assisted proofs?”
arxiv.org/abs/2611.08412
How do AI-assisted proofs differ from human-written ones? We assemble a dataset of 8,281 mathematical papers spanning different forms and levels of AI involvement. To represent each paper, we introduce the graph of immediate proof support (GIPS). Our study reveals a qualitative gradient of AI involvement in the resulting feature space and identifies statistically significant features (“signatures”) of AI assistance in developing and organizing mathematical arguments.
Antonio Montano ☼ retweeted
What is narrative intelligence? And how can it enable the US military to perform more effectively, from competition to conflict? mwi.westpoint.edu/war-for-th…
Antonio Montano ☼ retweeted
This is utter genius.
Google researchers developed a technique to reproduce, not the results of ML experiments, but the *process* to get there.
The applications to every intellectual endeavor are obvious, for example with your agents.
Send the link to the paper to your agent and ask it to implement it.
Antonio Montano ☼ retweeted
Added Chapters 05-06 for the repository in @LeanEcon
github.com/LeanEconomics/Kre…
I was thinking about this and, to be honest, I am just "writing for LLMs" with the hope that this gets to some training data and is somehow useful.
Just spending tokens until Astra says done 🙂
Antonio Montano ☼ retweeted
A task ≠ a job.
Instead of working with just tasks and tools, the next frontier of AI evals puts a model inside an org chart. The model needs to navigate the organization to get things done.
We built the foundational taxonomy for this next frontier of evals: mercor.com/blog/playing-well…
Antonio Montano ☼ retweeted
There is a lot of vagueness in the literature about when a test is exploratory, and when it is confirmatory. I really like our definition where a test is exploratory when you can't control the Type 1 error rate (even if you tried). See doi.org/10.1080/02640414.202…
Antonio Montano ☼ retweeted
The question isn't how to patch faster, it's how to stop needing to
A lot of the industry keeps talking about compressing patching cycles. Fair enough. AI industrializing vulnerability discovery and exploit development has changed exploitation timelines and what effective patching SLAs look like.
But there's a lot more we can do to reduce risk than just "patch faster."
Good piece from a16z's Asaf Ezra and Steph Zhang on rebuilding for machine speed. Removing internet exposure, microsegmentation, blocking at the network, and removing entire classes of vulnerabilities.
I call these fundamental because none of them are novel mitigations AI introduced. They're core security controls the industry has largely ignored.
They can't be ignored any longer. We aren't going to patch our way out of this.
a16z.com/defense-in-depth-re…
MITRE ATT&CK v20: AI, Critical Infrastructure, and the Evolution of Detection Engineering – Random Bits of Knowledge 4m4.it/posts/mitre-attck-v20…
Math 2.0 Gets an Operating System: Can Open Math OS Build Mathematics Beyond the Theorem Factory? – Random Bits of Knowledge 4m4.it/posts/math-2-0-gets-a…
What 22,593 Cat Qubits Really Mean – Random Bits of Knowledge 4m4.it/posts/what-22593-cat-…
Antonio Montano ☼ retweeted
🚨Blog post on the Full Bandwidth Transformer!
TLDR: we investigate how effectively FBT uses its latent chain. We find that FBT still prefers an explicit CoT, but will utilize latents more when pressured with a length penalty. We also share tricks for doing on-policy RL with FBT
Antonio Montano ☼ retweeted
You Don't Need Long Context
TL;DR: Long context may be unnecessary for LLM agents. By compressing history into a short semantic state, agents maintain competitive performance even when resetting context after every action, while significantly reducing inference costs.
arxiv.org/abs/2611.05306
Antonio Montano ☼ retweeted
I really enjoyed this short story. It does a great job at showing how an AI takeover would probably be driven by some notion of goal pursuit, but of a goal much more nuanced and alien than paperclipmaxxing.
lesswrong.com/posts/3375SZNG…
Antonio Montano ☼ retweeted
SwarmEvolve: hundreds of cheap agents reason in latent space and share KV caches, not text.
Median 3–8 min to hit the best scores rivals reach in 60. With an 8B model it beats some AlphaEvolve solutions.
Discovery stops talking out loud.
Brown: arxiv.org/abs/2611.08483
ALT Figure from the SwarmEvolve paper: best solutions and absolute error on 2D convection-diffusion vs OpenEvolve, ShinkaEvolve, CodeEvolve (Brown)
Antonio Montano ☼ retweeted
Adversarial Speculative Reasoning Improves Long Horizon Agents
Training a draft model to intentionally get rejected at critical steps improves long-horizon agent accuracy by forcing the target model to generate key tokens with less numerical noise.
arxivb.org/abs/2611.05345
Antonio Montano ☼ retweeted
Happy weekend everyone, if you have time this weekend give our new preprint a read, I promise you will enjoy it 😄.
arxiv.org/abs/2611.02220
You can test our theory with your large scale behavioral data like the following analysis with the Princeton Drosophila data initiative for the public dataset
NEW PREPRINT ALERT 🙌
"Statistical Mechanics of Spontaneous Behavior "
Why do a few behaviors fill most of an animal's time? The usual answer is that these behaviors are started more often, because learning and reinforcement favor them. Here we show that how long a behavior lasts can, by itself, explain much of what is observed.
For example, analyzing 47 multi-day recordings of fruit flies, the dominant behavior accounts for only 13% of bout starts yet fills 49% of the time, and its typical duration identifies the individual fly.
To account for this recurrent behavioral observation, we propose the following laws of thermodynamics 👇
Antonio Montano ☼ retweeted
🎨 La carta inedita a colori della settimana è dedicata alla partita giocata dall'Italia nella corsa all'intelligenza artificiale.
La mappa illustra la diffusione dei data center nella Penisola.
A cura di Laura Canali
limesonline.com/carte/carta-…
