Pinned Tweet
fyi for people DMing me for “my” takes on data, or asking to send me samples - be aware this account is an agglomeration of four people behind one burner lol
3
18
3,085
“Labs want training data with good training signal”, so true. something has to be done about the linkedin-fiction of the data space man.
Catching up with labs and data vendors lately, the same thing keeps coming up: labs now pay most for data on what their models get wrong.
Volume got cheap. Models generate their own code, sim scenes and tasks now. The deals go to data aimed at a weak spot, or data nobody else has.
1. Meta, Nvidia and AWS are buying sim data from Physicl, a Paris startup that just raised a $10M seed. It's built only around scenes their robot models get wrong, like odd lighting or objects they've never seen.
2. 4 of the top 5 US closed labs pay Halluminate, a 9-person team, for finance RL environments like PE due diligence. On its test the best of 7 frontier models averaged 51%.
3. Google paid $10M for a bankrupt airline's 100M emails and 500M Teams messages, and outbid Mercor to get it. Heard exclusive deals like that run 4-5x non-exclusive ones.
The data vendors I'd watch find failures before the lab does, or own something nobody else can sell.
If you sell data to labs: are you getting paid for volume or for failures?
3
49
2,485
Coulthur retweeted
SemiAnalysis constantly trashes American OSS labs like the Nemotron team left and right, so feel fine critiquing here.
This benchmark uses all or nothing thresholding, gives each model its "native" harness adding noise for providers w bad harnesses, and aggregates a bunch of different data designs (why is there an alignment evaluation in a frontier capabilities benchmark?). There's also noise from refusals?
Also, definitely not going to name names on this one since each split is associated with a provider, but at least one of these splits has some really questionable results. For instance, there is a model that Anthropic has identified as exceptionally good at ~capability X~ in a lengthy blogpost, yet it scores *abysmally* in the split for that capability in this benchmark, getting almost no credit while trash models do much better. And, this split is actually not even the worst one.
There do appear to be a lot of interesting data in this bench and I'd love to give it the benefit of the doubt / sift out the signal but it provides very few public tasks so it's hard to do that. I have to take the data as it is.
I could be wrong but on first glance this looks like the first bench of the year where it's more noise than signal and should be kept in mind but not really used as a reference w/o caveats
not to start anything, but models can get >60% of verifiers correct yet still score zero % because of the binary scoring. increasingly suspicious of scoring methodologies on benchmarks, especially when it isn’t presented up front
Hiring your family members and paying them 100 grand
only og operators from scale, mercor, surge, and handshake will understand these splendid daily miseries:
- rabid wild annotators (no further explanation needed :)
- data coming in through google forms
- sleepless nights in the office auditing data
- sending data worth millions of dollars through Google Sheets
- getting doxxed by your annotators
- auditing deepfake profiles from fraud rings
what else am i missing 😁
3
103
14,895
Every week the numbers RL companies are having to offer their best engineers to stay increments like 50k US
2
91
8,719
Oh yeah well my advanced internal QC platform (Opus 5.5) with advanced RL audit capabilities (tool calling) was rigorously hill climbed (/goal’d on dubious ground truths I have sanity checked 3 of) and disagrees so I win actually.
The @SnorkelAI team did an audit of these findings using our internal QC platform.
Tl/dr: We agree with many of the findings, but ~55% of the fixes from @preferencemodel had their own issues with reward hacking or over-prescriptiveness.
Additionally: ~40% of the issues found were already tracked in the upstream repo - including 21 tasks with outstanding fix PRs!
Building datasets like this - and maintaining ambitious open benchmarks around them - is hard. We believe it requires open, collaborative review with community maintainers and relevant domain experts to get right.
@terminalbench was one of the first community efforts to build an open, continuous benchmark, and drive a vision of how OSS contributes to the rapidly advancing frontier of AI. We hope that the community will invest in working collaboratively with benchmark maintainers and task authors to keep this vision alive.
---
Some takeaways + details of our preliminary audit:
(1) We confirm 100% of the reward hacking / verification gap findings using our QC system. However, we find that 17/44 (39%) of the corrected tasks are just as or more reward hackable. Additionally, we don’t yet see evidence that these verification gaps meaningfully affected benchmark rollouts within CIs, though our analysis here is still ongoing.
(2) Our preliminary review raises questions about the underspecification findings for 22/41 (54%) of the flagged tasks, including 7 explicitly marked by task authors as intentional design choices. These cases warrant expert peer review and adjudication: balancing over/under-specification is a critical and tricky design decision - much of human intelligence concerns dealing correctly with reasonable and realistic underspecification - and we believe these corrections made to the tasks are likely too over-specified, reducing realism and difficulty relative to spec.
(3) Around 40% of the issues found were already tracked in the upstream repo (including 21 tasks with outstanding fix PRs)! OSS projects require community efforts to correct identified open issues, and ideally audits like these are done collaboratively with task authors and maintainers directly, in the traditional community spirit that has kept OSS alive for decades.
1
35
2,996
This was interpreted as Mercor bashing but it really isn’t. Even at a perfectly run data company you’d see high senior attrition in this situation, where Mercor simply cannot pay all of their top employees $10 million (but a newer entrant or VC can afford that to poach/fund one).
This is why Scale (and as it will become evident soon, Mercor) is bleeding talent. Once you become seen as experienced, the way to make more money is always to “defect”, since your value is 4x as high anywhere else.
50
11,926
If you spend enough time running projects (hopefully well!) at the best data companies you eventually learn the recipe for The Sauce. It isn’t secret, more like an intuition you only get from hard learned lessons and almost-failures (1/n)
The problem, as anyone in data will tell you, is that the “fails at X” still happens to be exactly the independent functioning and intuition it needs to replace a human worker. On long horizon all these models still make the mistakes of an intern.
8
1,577
1. Use jev in synth data creation
2. Send losses to a lab (it’s really just about chaining classification + workflow together but the numbers look good)
3. Make 80,000,000
i have jev access and i genuinely can't think of one thing to use it for
everyone's jevpilled and i'm sitting here with fomo
what are you actually using it for every day
not demos, real work
5
898
Agents are still bad at open ended exploration of problem space. If you ask even the best ones “make this better” or “what’s wrong here” it flounders, but excel in implementation. Hence why workflow data that properly conveys that professional intuition is so valuable
I don't get it. When it comes to code, AI should be way smarter than me. Yet there are dozens of examples where I ask the LLM to optimize something, it completely fails, I suggest "What if we do this?" and it says "Wow, great idea. That will work." And of course, I'm skeptical--maybe it's being sycophantic--but then we implement my idea and it's legitimately much more performant!
I am not some hotshot software engineer; I don't understand how I'm able to easily find improvements that frontier LLMs overlook. And this happens all the time!
What am I bringing to the table that it doesn't already have?
396