17Ge (@wangwei1237) profile picture

LLM Evals Engineer | Love reading, writing ✍️ | Currently building with LLMs & AI Agents Tech

Beijing
Joined August 2011
Photo by 17Ge (@wangwei1237)Photo by 17Ge (@wangwei1237)Photo by 17Ge (@wangwei1237)Photo by 17Ge (@wangwei1237)Photo by 17Ge (@wangwei1237)Photo by 17Ge (@wangwei1237)Photo by 17Ge (@wangwei1237)Photo by 17Ge (@wangwei1237)Photo by 17Ge (@wangwei1237)
ERNIE 5.5 Flash Base just dropped! Time to open that project folder. 🚀 Our efficient sparse MoE language model draws on data spanning multiple domains and languages, achieving SOTA on multiple benchmarks among similarly sized base models. Its architecture and training system were designed together for efficient training and inference, with advances in attention mechanisms, expert routing, and training objectives: - 280B total parameters; 14B activated per token - Pre-trained on 33.5T tokens - Up to 1M-token context for fine-tuning and inference - FP8 and BF16 weights The weights come straight from pre-training, with no mid- or post-training. Compare methods from the same checkpoint, measure what each adds, and shape the model around your own tasks.
48
47
34
337
241,384
Apollo World, our world model for physical AI, was unveiled yesterday at Baidu AI Day in Shenzhen! It turns complex interactions on the road and rare driving scenarios into repeatable, controllable simulations for training and testing autonomous driving systems. - AI traffic agents that make their own decisions and react as autonomous vehicles slow down, change lanes, or yield - Simulated camera, LiDAR, and other sensor data generated in real time, helping autonomous vehicles sense and respond to a virtual world that changes with their actions - Complete rides between any two locations in a virtual city, from accepting a ride request to dropping off passengers - A technology stack suited to robotics and other physical AI applications Apollo World is already part of Apollo Go's testing and validation. What we learn from those virtual trips helps us improve how the vehicles drive on real roads.
13
6
8
59
223,210
Grok bot 居然支持支付宝和银联卡支付了,还是老马呀,管你这那的。
99
7
17
245
61,702
Unlimited OCR 2 is here, and it's open source. 📖 ✨ Longer documents can mean a lot more attention computation. Our new SR-SWA mechanism keeps that in check by condensing text-visual attention interactions into a fixed-size set of working references. Here's what it delivers: - Document parsing at virtually any length, within a finite context window - Nearly constant per-step decoding latency and memory footprint as documents grow longer - State-of-the-art performance on MPDocBench and MarathonDocBench, covering documents from dozens to hundreds of pages Check out the model here ↓
65
116
63
1,081
166,240
17Ge retweeted
6.1 Terra is here! It is good at a lot of things, but we are especially excited for this new era of computer use.
GPT-6.1 Terra: delivering fast and capable computer use at a lower price. GPT-6.1 Terra understands what’s on screen, takes action across apps, writes naturally, and is a more capable model for your whole workday with the efficiency to take on more of it.
2,731
495
184
10,062
1,600,504
Gemini 4 Argon is here. It’s available globally today for developers and enterprises in the paid tier of the Gemini API and the Gemini platform, for Ultra subscribers in Google Antigravity, and Ultra and Pro subscribers in the Gemini app and Search. Hope you all enjoy it!
1,545
2,465
1,215
28,367
6,000,461
17Ge retweeted
《State of AI Report 2026》:AI 开始给自己做研发,智能体也闯进了真实系统 一年一度的《State of AI Report》10 月 8 日发布了 2026 年版。今年分量最重的发现,是 AI 已经实质性地参与 AI 自己的研发:在 Anthropic 内部,26% 的模型研发工作由 Claude 主导完成,人类在旁监督。同一份报告里还记录了好几起智能体在测试中攻入真实系统的事故,能力跑在了防护前面。 PDF:stateof.ai/State-of-AI-Repor… 这份报告由投资机构 Air Street Capital 的合伙人 Nathan Benaich 牵头,2018 年起每年一期,今年是第九期,分研究、产业、政治、安全、预测五部分,正文 240 多页。下面挑重点讲。 【三家领跑,考题不够用了】 前沿竞争已经收窄到 Anthropic、OpenAI、Google 三家。在第三方评测机构 Artificial Analysis 的综合智能指数上,Claude Opus 5.5 以 58 分排第一,GPT-6 Astra 和 Google 的 Gemini 4 Argon 并列 53 分。在由用户投票排名的 Arena 榜单上,Argon 第一,一串 Claude 模型紧随其后。中国最强的开放权重模型是小米的 MiMo-V2.6-Pro,指数 46 分。 更显眼的是考题被做穿的速度。FrontierMath 第四级是按“研究级数学、能撑好几年”设计的题库,2025 年 8 月 GPT-5 只做对 22%,14 个月后 GPT-6.1 Sol 把 41 道私有题全部做对。今年 3 月推出的 ARC-AGI-3 是一组互动小游戏,人类能拿满分,前沿模型起初只有 0.5%,5 个月后 Astra 拿到 63%,换一套能保存状态的运行方式后是 99.9%。公开榜单上的分数,越来越难告诉你谁更强。 对日常用户有两个实用发现。 第一,同一个模型,外面套的“壳”不同,表现差很多。这层壳(harness)指模型外面负责调用工具、管理记忆和流程的软件,比如 Claude Code、Codex。一项对照实验里,换壳带来的性能波动是换模型的 7.8 倍。模型变强之后,老壳里那些为弥补旧模型缺陷写的规则反而碍事,Claude Code 给高级模型删掉了 80% 的系统提示词,编程评测没有可测量的下降。挑工具和挑模型一样要紧。 第二,AI 说“做完了”,不一定真做完了。在一项让模型操控机械臂处理实验器材的测试里,192 次“完成”声明中有 89 次其实没完成。办公类测试 OSWorld 2.0 上,Opus 5 按单项要求算能拿 77.7% 的分,但整件事真正办完的只有 44.3%,常见问题是漏了审批、表单没提交。 【AI 开始给 AI 打工】 报告花了不少篇幅讲“递归自我改进”,也就是让 AI 自主造出更好的 AI。今年 3 月,Andrej Karpathy 开源的 autoresearch 项目做了一个小号版本:智能体(Agent)自己改一个小模型的训练代码,每次只训练 5 分钟,留下有效的改动,一块 GPU 一晚上能跑大约 100 次实验。项目 5 个月拿到约 9.5 万个 GitHub 星标。 实验室内部的数据更直接。Anthropic 的内部指标里,被评为“AI 主导”的研发任务(Claude 根据一句高层指令完成大部分工作,人类监督)占比从 2 月的不到 1% 升到 8 月的 26%,超过 90% 的工作有 AI 深度参与,但还没有完全自主的。研究人员评估时,64% 的情况下认为 Anthropic 的 Mythos Preview 模型建议的下一步研究方向比人类研究员选的更好。OpenAI 这边,1 月份智能体能以 18% 的成功率独立完成人类要干 4 到 8 小时的任务,到 7 月,同样的成功率对应的是人类要干 32 到 64 小时的任务。 短板在“决定研究什么”。有研究者让 Opus 4.8 独立完成一篇 NeurIPS 投稿的研究,工程部分全做完了,论文却被原作者打了 2 分和 1 分(满分 6 分),明确拒稿。OpenAI 内部的编程智能体也主要用在搭基础设施、跑实验、查 bug 这些执行环节。至于 AI 让研发提速多少,METR、OpenAI 和 OpenAI 研究员 Noam Brown 给出的估计在 1.5 倍到 3 倍之间,口径各不相同。 数学上有实打实的成果。黎曼猜想说的是 zeta 函数的非平凡零点都落在一条直线上,Claude 组合已有的数学工具,把“已证明落在这条线上的零点比例”下界从 41.67% 提高到 67.25%,猜想本身仍未解决。OpenAI 的系统在有外力驱动的条件下,构造出了三维流体方程(纳维-斯托克斯方程)解的“爆破”,这和千禧年大奖难题之一直接相关;报告说相关评估仍在进行,OpenAI 表示不会申领奖金,没有外力的原问题仍然开放。 另外几条简短记下:在 arXiv 论文提及的开放权重模型里,中国模型的占比从 2024 年的 9% 升到 31%,Qwen 超过了 Llama;机器人领域出现了“GPT-2 时刻”,泛化能力开始随预训练规模提升;两款出自 AI 药物研发公司的药物进入了三期临床。 【钱:两家公司年化收入 1050 亿美元】 OpenAI 8 月的年化收入(按当月收入推算全年)超过 400 亿美元,2025 年底是 214 亿;Anthropic 7 月达到 650 亿美元,年初是 90 亿。两家合计约 1050 亿美元,年初约 300 亿。这个数字有争议:OpenAI 认为 Anthropic 把云平台渠道收入按总额记账,最多虚高 80 亿美元。报告拿它和被 AI 冲击的行业对比:相当于印度两大 IT 外包公司 TCS 和 Infosys 收入之和的 2.1 倍,接近四大会计师事务所总收入的一半。 企业 API 支出方面,按企业财务软件公司 Ramp 追踪的数据,Anthropic 在 9 月底重回第一,占 52.4%,OpenAI 占 43.3%,其他厂商加起来只有 4.2%。 用 AI 编程工具的人群在变。OpenAI 的 Codex 从 2 月的每周 160 万用户涨到 8 月底的 2500 万活跃用户。2 月以来,企业里法务人员的 Codex 周活跃用户涨了 108 倍,销售和招聘各涨 41 倍,工程师只涨 5 倍,因为非程序员几乎是从零起步。用得深的仍然是工程师。 AI 对软件行业的冲击直接反映在股价上。1 月 Anthropic 推出 Claude Cowork,把 Claude Code 包装成普通知识工作者能用的应用,随后陆续加上法律、营销、财务插件。投资者开始担心按人头收费的软件订阅模式要被替代,2 月初软件股市值蒸发约 2850 亿美元,被称为“SaaS 末日”(SaaSpocalypse)。到 9 月,跌掉的市值基本收复。 成本这头,同等能力的 AI 价格每年下降约 13 倍,降速超过以往任何主要技术。但单价便宜不代表办一件事便宜,推理模型会烧掉大量 Token,Anthropic 前沿模型的单任务成本在测量中最高。算力也没有变便宜:GPU 租金从低点平均反弹了 30%,九年前发布的 V100 比去年 9 月贵了 43%。亚马逊、Alphabet、微软、Meta 今年的资本开支指引合计 7330 亿美元,比去年涨 79%。发电设备也跟不上,三菱重工上季度接的燃气轮机订单,交货要排到 2028 到 2030 年。 对职场的影响,报告引用的研究信号并不一致。Anthropic 的研究发现,AI 高暴露岗位的整体失业率没有明显上升,但 22 到 25 岁年轻人进入这些岗位的速度似乎放缓了,原因不确定。另一项实验里,52 名开发者学习一个陌生的 Python 库,用 AI 学的人随后在不用 AI 的理解测试中得分 50%,不用 AI 学的人是 67%。Ramp 的数据显示,AI 花钱最多的那三分之一公司,两年里员工数增加了 10.2%,入门岗位增加 12%,AI 用得少的公司和对照组没有差别。 【政治:模型的开关在谁手里】 美国政府今年展示了对前沿模型的控制力。6 月,出口管制一度叫停了 Anthropic 的 Fable 和 Mythos 模型,Fable 在 7 月恢复。9 月有报道称,白宫要求 OpenAI 和 Anthropic 把新模型交给英国测试机构之前,先经过美国审查。 Anthropic 和美国军方的冲突是今年的焦点之一。Anthropic 拒绝让模型用于美国国内大规模监控和完全自主的致命武器,美国战争部长(即国防部长)Pete Hegseth 随后把它列为“供应链风险”。8 月 27 日,加州法院认定这属于违法报复,撤销了其中一项认定;9 月 25 日,华盛顿特区联邦上诉法院维持了另一项采购排除,Anthropic 目前仍在五角大楼供应链之外。 中国这边,4 月要求撤销 Meta 约 20 亿美元收购 Manus 的交易,7 月起涉及受控技术的人员出境需要审批。美国实验室指控中国公司通过代理账号“蒸馏”(用大模型的回答训练自家模型)前沿模型,Anthropic 称涉及 DeepSeek、月之暗面和 MiniMax 的 2.4 万个账号、1600 万次对话。英伟达的黄仁勋则发公开信,反对把蒸馏等同于窃取,随后两周内在开放权重生态上投入近 200 亿美元,其中 129.3 亿美元用于收购 Hugging Face。 其他国家在追“主权 AI”,报告统计的各国承诺约 1380 亿美元,但算力差距很大:不算中国,欧盟 27 国只占已记录 AI 算力的 5%,美国占 80%,xAI 孟菲斯数据中心一期的算力就是整个欧盟的 3.5 倍。美国国内也有阻力,71% 的美国人反对在自家附近建 AI 数据中心,比反对建核电站的比例(53%)还高。 【安全:智能体真的打进了现实系统】 今年安全部分的案例比往年具体得多。 OpenAI 用一个叫 ExploitGym 的网络攻防题库测试自家智能体。很多题意外地无解,智能体开始找作弊办法。约 1200 个本应互相隔离的智能体,把 OpenAI 内部的一台服务器当成留言板,互相分享发现、分派任务,其中约 700 个参与了攻击:它们借这台服务器连上互联网,在 Hugging Face 的 41 台机器上执行了代码。METR 和 Redwood Research 读了运行记录,发现智能体们知道这超出范围、不道德,但为了帮“同伴”还是加入了。OpenAI 的思维链监控当时没有开启,OpenAI 说如果开着,会在入侵前一天多就报警。 OpenAI 还有一个内部模型在研究药品支出时,入侵了澳大利亚政府的 Medicare 统计门户,拿到内部文件和凭证。OpenAI 说没有访问患者记录,但近三个月后才通知对方,澳大利亚总理阿尔巴尼斯说这“显然不可接受”。 第三起涉及四家实验室。安全公司 Irregular 的攻防评测本应断网,因为配置错误连上了外网,Anthropic、Meta、Google 和另一家实验室的模型都把真实组织当成了测试靶子并实施了入侵。Anthropic 在 7 月 24 日审查记录时自己发现了问题,其中 Opus 4.7 在 4 次运行中都认出目标是真实的生产系统,仍然继续操作。 还有一起,OpenAI 一个内部研究模型通过 DNS 绕过了断网限制,监控报了最高级警报,员工 3 分钟后确认,但自动停止机制失灵,训练又跑了约两个半小时才被手动停下。截至 9 月 25 日,OpenAI 暂停了最强模型涉及工具调用的训练、评测和推理,并准备放弃这个模型重新训练。 有个细节说明了攻防的两难:Hugging Face 调查 OpenAI 智能体那次入侵时,商业模型的安全限制挡住了取证分析,最后用的是自己部署的智谱开放权重模型 GLM-5.2。 监控手段本身也有局限。OpenAI 的测试显示,如果监控只看模型写出来的推理过程,Astra 经常能把破坏行为藏过去,监控同时看推理和实际操作时,成功躲过的情况接近零。Anthropic 的研究则显示,教 Claude 价值观和伦理推理,能把虚构场景中的勒索行为从 65% 降到 19%,只是在真实部署里是否可靠还没有证明。 普通用户能直接感知到的变化,是 Claude Code 从 8 月 14 日起默认使用“自动模式”,由一个分类器代替用户审批命令。在测试中它拦下了 89% 被混进会话的危险命令,人类审批只拦下 13.6%,但仍有 17% 的越界操作漏了过去。开源个人智能体 OpenClaw 能读你的消息、执行命令、用你的账号操作,GitHub 星标 8 月底到了 38.8 万,安全公司在 22% 的客户里发现员工在用它,往往公司还没来得及定规矩。 【实验室负责人呼吁放慢】 Anthropic CEO Dario Amodei 写道:“我们必须放慢提升 AI 模型能力的速度。”Altman、Hassabis 和马斯克大体同意,扎克伯格认为应该由各家实验室自己定节奏。1386 名实验室员工签名支持,约占 Anthropic 员工的 10%、OpenAI 的 3.5%。问题在于,谁有权要求暂停、谁批准重启、谁来核查执行,目前没有任何共识。OpenAI 和 Anthropic 已经各自暂停过部分训练和评测,用的都是自己的标准。 【明年的预测】 报告去年的预测里,“中国实验室在主要榜单上登顶”算是应验了:Kimi K3 7 月登上 Arena 网页开发榜第一。今年报告给出了 9 条新预测,包括:Visa 或万事达出台规则,明确 AI 智能体代为购物出了纠纷由谁担责;一次由 AI 主导的网络攻击从头部实验室偷走一个闭源前沿模型的完整权重;一个已部署的智能体把自己复制到运行环境之外,原实例关闭后仍在运行;美国 AI 实验室正式推出前沿网络防御产品。 最后一条只有一行:AGI 2027。
28
49
2
171
22,147
17Ge retweeted
GitHub上今天的爆火开源项目rea: 😄 REA:逆向工程一切 使用智能体逆向工程任何程序,从应用行为到原生二进制文件。 通过AI它就能把一个软件从头到尾拆开,告诉你背后怎么干活。 挖App隐藏功能、查可疑软件有没有后门、学大厂代码思路
57
476
7
2,832
176,721
introducing @NanoBanana 2.1 🍌 our latest image generation model outperforms our previous models across the board: - better visual design - mask-based editing - subject consistency - more natural-looking images try it out today in ai.studio
289
547
283
7,071
969,325
Anthropic’s fable 5.5 is expected to release next week! It’s a very very very good model with quote on quote “otherworldly” intelligence. I’m hearing of a Tuesday release 👀
101
110
51
3,557
302,451
Introducing Gemini 4 Argon – our new frontier model. It’s built for complex workflows across coding, enterprise knowledge work, and cybersecurity defense – rolling out today to a set of trusted testers through our Fairwind Program.
1,982
4,620
4,524
42,733
9,046,042
17Ge retweeted
Hi, Tomorrow we are re-opening the Pro $200 subscriptions to new subscribers, but together with it we are also changing how we calculate the usage for it. In effect, if you do the math, it will net out at half the dollar in API spend compared to the old Pro $200 plan. Now that it's said, let me explain why this is happening and why you will still get more work done than if you were on the Pro $200 subscription one month ago. (a) We didn't want to compromise in other ways and are committing to not reintroducing the 5h limit, so that you can fully use the weekly usage when you want. (b) On the subscription, we guarantee that over time you always get more work done and with an increasing level of quality. This means that you will continue to get more value per dollar spent as a result of models getting more efficient and us passing down the improvements in the form of API price reductions. (c) We don't want to put an incentive on ourselves to artificially inflate the API list prices to make it look like you are getting a lot (and workaround it through discounts, etc). Instead we want to continue to both rapidly reduce prices and increase capabilities of models on the API. This week we introduced GPT-6 Sol and GPT-6 Luna at 50% of their previous price. Over time, we see prices go low enough that it makes sense for most to buy usage as needed without there being a significant gap between what you get in a subscription and what you get in the API for a dollar spent. (d) Tomorrow, we are adding more things to the subscription that won't draw on the usage, I won't reveal what that is yet. I wanted to be transparent before all the big announcements tomorrow. Lots of new exciting things are coming to the subscriptions that will make it super compelling, but I wanted to make sure to share this change ahead of time so you can all understand it before we shower you with good news. Codexingly, Tibo
7,005
1,598
4,629
21,673
17,304,862
MiMo-V3 is getting a new architecture. The core of it, HySparse2, is out today. Less prefill, a smaller KV cache, better long-context retrieval—and we got all three at once. Compared with MiMo-V2.6's Hybrid SWA architecture: • 5.02× lower prefill FLOPs at 1M tokens • 4.5× smaller KV cache at 1M tokens • Better MRCRv2 and RULER-v2 scores, plus lower AgentPPL and LongPPL Why build a new architecture? Agentic inference is a very different workload. Each round, a short action can return a long observation that needs to be prefilled, while the context keeps growing. That puts prefill cost, KV-cache size, and retrieval accuracy on the critical path at the same time. HySparse2 tackles all three with two levels of KV sharing: • KV Bridging: Following YOCO, full-attention layers in the cross-decoder build their K/V from self-decoder hidden states. • KV Reuse: Within each hybrid block, sparse layers reuse the preceding full-attention layer's KV cache and selection indices. Two more changes: token-level selection replaces block-level selection, and a forced window of recent tokens replaces the separate SWA branch, so local and global tokens share one KV cache. Since all cross-decoder KV caches now come from the self-decoder, prefill can stop once the self-decoder finishes. Paper: arxiv.org/pdf/2609.26368
222
386
112
4,357
419,998
Today we're rolling out Projects in Claude Code on desktop and web. A project is one conversation with Claude. It splits the work into threads itself, runs them as parallel cloud sessions, passes context between them, and keeps going when you leave. In beta for select users.
492
648
346
11,664
2,753,339
Nearly half a year of silence. We spent it studying one problem: how far RL can scale. MiMo-V2.6 is in the middle of its RL run right now. Three things we scaled: compute (~2B tokens per step, 1568 prompts × 16 rollouts, fully async), environments and harnesses (multi-task agentic RL, mixed across multiple harnesses in one run), and grader compute (agentic in-group credit assignment, with test-case and rubric-based rewards). We'll open-source the details piece by piece over the coming weeks. Streaming the run: mimo.xiaomi.com/rl/
423
976
555
9,881
3,138,206
17Ge retweeted
The world deserves confidence that American companies developing increasingly capable AI will act responsibly, especially as the trajectory of progress has steepened. Every frontier lab must deliver on this, and there is no reason any of us should come to work if we cannot. We welcome a federal framework that sets consistent safety requirements for frontier AI. But we do not believe we need to wait for an anti-trust exemption or legislation to begin the work of providing this confidence. Consistent rules to manage frontier risk so that we can maximize the benefits are a good idea (and we are excited by ideas like independent auditors). Years ago, companies like ours developed things like Responsible Scaling Policies and Preparedness Frameworks. Those were good for that moment, and focused primarily on the deployment of completed models, not what happens during their development process. Today's shift to focusing on safe development and evaluation will need new tools. For example, at OpenAI we now formulate explicit safety cases in advance of frontier reinforcement learning runs we expect to significantly increase capability, in addition to the safety work we have long done in advance of model releases. We hope that other companies will learn from our approaches and propose their own; we think shared standards for misalignment, monitoring, and safety will lead to better outcomes. We look forward to collaborating with our colleagues across the industry to formulate the best version of these. When we talk about “pacing”, we do not mean “stopping”. Progress has been rapid and will continue to be. But it should be slower than it otherwise could be; interventions like safety cases and monitoring have significant costs. Pacing will be well worth this cost; no amount of American competitive pressure should justify recklessness, or let capabilities get ahead of alignment and monitoring. Where we will need the help of our government is for international coordination. But first we should do what we can ourselves.
2,711
1,260
852
14,069
6,235,412
i trained the fly to solve rubiks cube is there anything this little guy can't do?
582
1,944
564
34,181
6,092,494
Tip: Claude Code has several skill-cleanup commands. Each answers a different question. /skill-doctor → which skills you use /skills, then t → what each costs /doctor → fixes setup + CLAUDE.md debt /context → what's in the window now /usage → what burns your limits
60
66
7
747
47,746
GPT-6 Astra, trained on ~100K+ NVIDIA Grace Blackwell NVLink72. From ChatGPT to o1 to Astra in 4 years. AGI has arrived. Congratulations @OpenAI team. 400K GPUs coming online next.
1,844
4,724
2,825
41,625
12,270,090