So latent state communication sounds cool
But if you actually think about it, this is fundamentally encoder-decoder architecture.
Where encoder encodes and the hidden states are passed to the decoder to do generation.
Just replace the encoder here with a bigger LLM (753B) and decoder with a smaller one (4B)
Infact most VLM work like this, the only difference is the size of encoder and decoder is reversed
meet @mostik_ai!
what happens when you put 12 PhDs in one room for four months? first place on the ARC-AGI leaderboard, which I can't say much about while the competition is still running. and this, which I can.
everyone's arguing about whether open models will catch up to frontier models. we think it's the wrong question. here's the one we pose: why does a frontier model have to generate your answer at all, when the only thing you need from it is the reasoning?
we do this by enabling models to communicate in latent space. through our protocol, hidden states pass straight from a frontier model into a small one running on your infrastructure -- no text between them, and neither model is fine-tuned. two models from different families, sharing reasoning, both left untouched.
how do we know it works? we tested it on a setup where a 753B model reads the problem, and a 4B edge-class model writes the answer. with this approach, we get results 80% as accurate as the frontier model, but at 20x faster performance.
we're committed to preventing frontier model lock-in and are already partnering with inference providers to accelerate open-weight adoption. we've done this between 15 of us, in four months, 12 PhDs and a Fields medalist, backed by @generalcatalyst
WIRED has the first external account of the company and the work: wired.com/story/russian-star…
full writeup, the setup, and all the numbers: mostik.ai/read-more
30
Thankful to whoever came up with the “Bought earlier” tag on @letsblinkit, surely the PM knew what they were doing. Such a simple feature but makes the user’s life so easy.
1
2
20
This is so tempting to impulse buy
We built a small biped robot you can teach new tricks to.
Train it in simulation, run it on the real thing. Meet Microduck 🦆
$399, shipping before Christmas.
pollen-robotics.com/microduc…
github.com/pollen-robotics/m…
30
Both Anthropic and Openai have started having Applied AI roles in India.
In reality these are Solutions Architects for Enterprise Accounts and Scaling Growth in India.
These are not Applied Research roles.
Had a couple of “Applied AI” and Sales folks from Openai at our office last week.
1
4
113
Great short explainer if you don’t know where to start
Claude's watermark probably doesn't work how you think. As the CTO of GPTZero, I'll explain how Anthropic, Google and OpenAI are building text watermarking in this brief explainer and whether it can be defeated.
Almost all forms of watermarking that are fast and cheap enough for a frontier lab have the same formula, following the KGW method:
In generation:
1. Let's say you've generated n tokens so far. Take those n tokens + a secret key to generate a random hash
2. Use that hash to randomly reweight the probabilities for the n+1 token, and then sample from that new distribution. In the simple case, you could split 50% of all English words into a green or red set based on your hash, and boost the probability of words in the green set.
For watermark detection:
1. For each token, see if it was in the green or red set.
2. To do this, recreate the hash based on the secret key and the text preceding the current token. Then, recreate the green and red set of words.
3. Once you've checked all the words in the text, if the next token is selected disproportionally from the green set more than 50% of the time, you claim the text has the watermark.
I can tell you want to ask the following:
1) Isn't it easy to mess up the hash if you paraphrase the text? The answer is mostly yes, however, you can use a statistical model to get your hash instead of a deterministic function (SIR, Adaptive Watermark). Since the entire watermark is probabilistic, this is fine.
2) Doesn't this make the text much worse? The answer is yes, it does - Yes, it does – but for most people, it's imperceptible (Google claims in human feedback study with 20,000 texts), since there are exponentially many ways to write the same paragraph. DiPmark does something more sophisticated to avoid shifting the text distribution on average. Of course, watermarks fail on short text or highly predictable texts like "2+2=4".
3) Shouldn't it be easy to figure out the green and red sets? The answer is no. You would need an exponentially large number of samples from the watermarker to reconstruct those sets exactly, but it's a risk if the detector is open to the wild (Watermark Stealing)
Still, there are couple challenges that a frontier lab needs to overcome:
1. Their watermark needs to work token-by-token because they are streaming their text to users. Many watermark methods plan sentences or paragraphs at a time, or change the text after its entirely written, in order to make their watermark robust to paraphrasers, and a frontier lab cannot afford to do this yet (SemStamp, PostMark)
2. If the secret key leaks, the watermark is busted. To avoid a large blast damage from this, you need to have a couple secret keys in rotation.
3. There are some texts, like code, that cannot be arbitrarily changed, otherwise the code will break. In those cases, the watermark needs to selectively change words in parts of the text that can tolerate synonyms (i.e. like variable naming) - see SWEET, EWD, Invisible Entropy.
4. They will need to educate their users on how to deal with false positives and false negatives of a detector, which is a big challenge (one we put a lot of effort into)
So, how do I see this playing out in the next 6 months?
1. If Anthropic releases the watermark detector publically, I think they defeat their own watermark. People find reliable watermark removal strategies by testing against Anthropic (AI detectors like GPTZero have an advantage here because they can train against these adversaries once they become popular).
2. If they keep the detector private to the government, like Google has done, it's "safer". However, there are some papers showing trained approaches that work robustly to zero-shot break watermarks without any data, simply because they try to write the text just like a human (Zhang et al. 2024, Watermarks in the Sand). Also, making your detector makes it battle-tested and stronger long-term (my experience).
3. In my testing, the watermarks don't survive intense paraphrasing (especially if you combine word choice and syntax attacks), or human text substitution (rewrite your AI text by plagiarizing human authors). The free paraphrasers I've tried have quickly bypassed Google Deepmind's SynthId for what it's worth.
4. All-in-all, frontier labs are likely okay with this because they expect most users to not attack the watermark, and also because they + European regulators likely don't care past a certain point - its good enough.
5. Overall, I think users of frontier LLMs will not really care about this, because 1) they don't realize watermarks are there, 2) EU will force everyone to conform, 3) this seems more like regulatory hoop-jumping than an earnest effort from frontier labs to expose LLM use
Lastly, people's first concern shouldn't be watermarking, it should be AI detectors!
If you're posting, "its not X, its Y!!", I don't think the watermark is going to make a difference :)
1
56
Prompt: write a post on insider trading without saying ‘insider trading’
I have, at any given time, between four and six remote part-time jobs at small and mid cap technology companies whose stocks I either own or am considering buying, and I do this not because I need the money, which I do not, but because the only way I have ever found to actually understand how a software business works is to be inside one, on a Slack workspace, in the standups, watching the sales pipeline get debated on a Tuesday morning by people who do not know I have a 4% position in the parent.
I started doing this in 2019, almost by accident, when a portfolio company I owned posted a $40-an-hour contractor role for a customer success rep and I applied on a whim, mostly to see how the hiring process worked, and ended up taking the job for four months. I learned more about the actual business in those four months than I had learned in three years of reading their 10-Ks. I learned which features customers actually used and which were vanity items in the marketing deck. I learned that the renewal team was understaffed in a way the IR deck would never acknowledge. I learned that the CEO sent personal notes to every customer above $50,000 in ARR, which was the single most important piece of information about the company and which appeared nowhere in any public filing. I tripled my position. The stock doubled in 18 months.
I have done this seven more times since. I have worked, at various points, as a part-time SDR at a vertical SaaS company in legal tech, a contract content writer at a developer tools company, a customer support rep for a small fintech, a freelance Salesforce admin for a healthcare software company, and a 10-hour-a-week implementation consultant for an HR tech company. The pay, in aggregate, has been somewhere around $90,000 a year, which is irrelevant. The information has been worth roughly $4 million in compounded gains across my portfolio, which is not.
My wife, who I love deeply, has stopped asking what I do for work. She has, over the years, developed a small mental model in which I am simultaneously unemployed, employed by six companies, and managing a seven-figure investment portfolio, and she has elected to leave the contradiction unresolved. My CPA has a similar resignation. He files the W-2s in a folder labeled “miscellaneous.” The IRS has, to my knowledge, never asked any questions, although in 2024 they did send a polite letter requesting clarification on why a man with my stated occupation was receiving 1099s from a Series C company in Provo.
This is, in the most literal possible sense, the work. The 10-K tells you what management wants you to know. The earnings call tells you what management has been coached to say. The internal Slack tells you what the company actually is, and the only way to read the internal Slack is to be on it, which requires getting hired, which requires being willing to do the kind of unglamorous, low-status, technically beneath-you work that nobody else who manages real money would ever consider. That unwillingness is, as it has always been, the entire reason it works. The information is sitting there. The companies are hiring. The job listings are public. Almost nobody applies.
1
35
Not unexpected, labs benchmaxxing to inflate model performance. Would be interesting if someone can run experiments to prove benchmaxing on OlmOCR and OmniDocbench.
Simply, Prompt with 80% of the document text token without image and if it completes the 20% correctly = benchmaxx
"When a metric becomes the target, it stops being a good metric" - Goodharts Law
last few days GLM-OCR has been trending after it claimed 95% on OmniDocBench, which is higher than Gemini-3-pro
in reality GLM-OCR is way worse than the story these benchmarks paint, lets see how
full disclaimer: ive been working in this space for the last 7 years with @nanonets
44
Rann retweeted
> be me, applied scientist at amazon
> spend 6 months building ML model that actually works
> ready to ship
> manager asks "but does it Dive Deep?"
> show him 37 pages of technical documentation
> "that's great anon, but what about Customer Obsession?"
> model literally convinces customers to buy more stuff they don't need
> "okay but are you thinking Big Enough?"
> mfw I am literally increasing sales
> okay lets ship it
> PM says there's not enough Disagree and Commit
> we need to disagree about something
> team spends 2 hours debating whether the config file should be YAML or JSON
> engineering insists on XML "for backwards compatibility"
> what backwards compatibility, this is a new service
> doesn't matter, we disagree and commit to XML
> finally get approval to deploy
> "make sure you're frugal with the compute costs"
> model runs on a potato, costs $2/month
> finance still wants a cost breakdown
> write 6-pager about why we need $2/month
> include bar raiser in the review
> bar raiser asks "but can we do it for $1.50? we need to be Frugal"
> spend another month optimizing to hit $1.50
> ready to deploy again
> VP decides we need to "Invent and Simplify"
> requests we rebuild the entire thing using a new framework
> framework doesn't exist yet
> "show some Ownership and build it yourself"
> 3 months later, framework is half done
> org restructure happens
> new manager says this doesn't align with team goals anymore
> project cancelled
> model never ships
> manager gets promoted to L8 for "successfully reallocating resources"
> team celebrates with 6-pager retrospective about what we learned
> mfw we delivered on all 16 leadership principles
> mfw we delivered nothing else
> amazon.jpg
What is it called when you are just high on your thoughts without any stimulants?
It is one of the most underrated and exquisite human experiences.
There is no restless urge to act, no sudden need to solve a problem. You are simply basking in it.
1
68
Rann retweeted
People are oddly consistent. Karma is just you repeating your patterns, virtues and flaws, until you finally get what you deserve
Rann retweeted
A fit body, a calm mind, a house full of love. These things cannot be bought - they must be earned.
Perhaps sometimes the only way to experience the beauty of things, is to think about things in a beautiful way.
333
















