It’s 2026 and frontier LLMs STILL hallucinate. Why? In our new ICML 2026 Position Paper, we offer a simple diagnosis and a constructive path forward.

jokes aside on why it took so long, this feature alone has single handedly improved my productivity by soooo much the past couple of months!

this is a really nice way to view AI progress. such a nice and clear graph!

my suggestion is more extreme than most I'm seeing here in the comments... but desperate times call for desperate measures, and as the 60k ICLR submissions show, the ML community has been too slow to rise to the challenge. the gist of my proposal is to institute two parallel submission tracks; let's call them Track A and Track B. at submission time, authors choose which track. in Track B, papers are reviewed and judged entirely by AI. This is not to say that it's an accept-all, AI-slop track: the conference still sets standards for acceptance, and papers that don't meet them are rejected. The difference is that those standards are enforced by an AI review process, not by human reviewers. each conference can design this process however it likes, based on the principles it values and acceptance criteria set by its human steering committee. It could choose to mimic existing reviewing practices (blind reviews, multiple reviewers, discussion, AC/meta-review), or invent something entirely different from scratch. I expect this process to improve empirically over time. in fact, the community can actively research better methods for AI review, test where they fail, and iterate from conference to conference. this process itself should be public and scrutible; the new open review, of you'd like. crucially, in Track B, there is no limit on the number of submissions. in Track A, on the other hand, papers are reviewed and judged by human reviewers, much as they are today. But there is a hard cap on how many papers an individual can submit to this track, to keep the demand for human attention under control. I have in mind a cap much smaller than what people are currently churning ou, say, something on the order of 1–3 papers per author per conference. The exact mechanism is a design question; the important part is that access to human review is deliberately scarce. are you a PI whose students typically produce 10–15 papers per conference? great. think hard about which ones would really benefit from human review, or which contributions you especially want the community to pay attention to. submit those to Track A, the rest to Track B. in the longer run, maybe this actually aligns incentives much better. e.g. two talented students could be incentived to join forces on one really ambitious project with their PI as author, instead of producing six incremental works between them without their PI as author. I don't think it's controversial to say the community would benefit from fewer, more substantial research contributions, and I think this has been true for years. AI agents coming for our peer-review process may simply be the tsunami strong enough to finally get enough stakeholders onboard to change the status quo.

The gap between frontier models' ability to derive clearly superhuman results and their absolute lack of ability to write about them in a human legible way is IMO the best current example of the "jagged frontier"

As an author of one of the 15 audited benchmarks, some personal thoughts on this: 1. There’s such a tight loop today between measurement and optimization, that high profile benchmarks materially impact what counts as progress in our field. This really raises the stakes on benchmarking work, and makes the difficult and not very glamorous work of re-examining existing benchmarks with critical (and impartial) eyes extremely, extremely valuable. I’m really glad to see @EpochAIResearch starting this initiative, congrats. 2. On the other hand, as someone working on frontier evals for the past few years, I want to emphasize how virtually impossible it is to put out a flawless benchmark. And it gets meaningfully harder the wider the scope of the benchmark is and the more agentic the task is. 3. I don't think it's helpful to view benchmarks as binary good vs bad, “flawed” vs “verified”. Auditing them, finding problems and improving on those problems is better viewed as a continuous process. In some sense it inevitably is, because some grading problems only become apparent when models become capable enough to utilize them, and are really hard to forsee in advance. 4. The wider the participation in this process will be, and the more parties that have a stake in these benchmarks, the better the outcome will be for the entire community. 5. This is essentially what we tried to do with SimpleQA Verified: we used OpenAI’s SimpleQA internally for some of our factuality work, realized it has a lot of issues, improved on them as much as we could at the time, and put the revised benchmark back out in the hands of the community. Based on my personal experience working with some of the other benchmarks audited by Epoch AI that were found as problematic, this can work just as well on them too.

this was sooo interesting and thought provoking to read, felt like brain food in a sea of slop

+100 for both takes (quoted tweet and article), but I'm curious whether anyone actually tested this empirically in a controlled setup? E.g., take people with different levels of expertise in a specific research area (undergrads → grad students → experienced researchers/PIs), give them the same research task + LLM + fixed time/token budget (say, produce a research paper that advances that area) and have experts blindly judge the outputs. Does the quality of what people can get out of the LLM scale with their domain expertise?

incredible moment makes me jealous of my math friends today, who can make sense of some of these proofs coding agents are a crazy productivity boost for me, but I've never actually had the feeling (yet??) that I'm reading something written by someone truly smarter than I am

seeing how really really smart people interact with really really smart AIs is quickly becoming one of the most interesting (and genuinely fun) ways to learn

I enjoyed reading this piece so, so, much. A tour de force of history, cognitive science and psychology to paint the current moment in AI in truly thought provoking ways. We really need more voices like in the public discourse on AI. Thank you @AlisonGopnik!

we know LLMs are poor at "knowing what they know," and RL is a natural place to intervene. in new work led by @pybeebee, we study a modified reward using Metacognitive Feedback (RLMF): completions where the model also accurately predicts its own performance get reinforced more, and we show that when compared to standard RL (GRPO) this gives much better results on the task of faithful uncertainty expression. check out the thread and paper for many more results! overall this feels like a promising direction, and I'm curious to see these types of techniques applied more broadly and to more generic problems.