Doing LLM Evaluation Research. Prev PhD @CUHKofficial, BS @PKU1898.
Hong Kong
Joined August 2017
- Tweets188
- Following344
- Followers327
- Likes351
(5/5) We report rounds, primitive actions, wall hits, and coverage. Some pairs differ in model routes, client versions, or prompts. Check out the project page to try the maze, change the wall-hit budget, or watch all 8 case clips: kennymckormick.github.io/blo…
41
(4/5) A wall follower cleared all 8 original loop-free mazes using only move outcomes. Low collision counts can also hide stalled exploration. Sol hit just 2 walls on a Hard hex maze and found no new cells in its final 581 rounds.
24
(3/5) Score those same runs with a 20-hit limit and the result flips. Vision clears 8/32 and text clears 0/32. On one hex maze, GPT-6.1 Sol + Codex reached the exit in both modes with 0 wall hits in vision and 3,823 in text.
46
(2/5) We compared 32 pairs on the same maps, starts, and goals. Vision agents got first-person views and action feedback. Text agents got only action feedback during play. Both could batch actions. Under the planning caps, text cleared 23/32 and vision cleared 13/32.
82
Share the stats of ICLR submissions in my batch, also 2 or 3 got desk-rejected. The top one is a benchmark.
5
1,820
We found Astra's pixel art generation genuinely impressive: not just downsampling, it keeps the key details in the image so the result still captures the essence of the original. Even better, its pixel art ability can scales up to hundreds of pixels.
The era of evaluating multimodal models with QA is over.
Use pixels instead. (0/7)
QA only tests high-level text reasoning. SVG and CAD rely on symbolic shapes rendered by external engines. Neither tells us if a model can directly control or read raw pixels.
We introduce PixelBench: a direct text-to-pixel probe where LLMs and VLMs generate and understand images pixel by pixel.
10
622
KennyUTC retweeted
为什么你花 2 万块买的旗舰智能电视，按遥控器还是卡得像十年前的安卓机？
昨天安全研究员 @log4j 把自家 3000 刀的三星电视 Root 了，扒出来的底层代码堪称年度草台班子大赏：
1. 电视后台每 500 毫秒截一次屏（连 HDMI 投屏看电脑桌面都截），打包发给近 400 家合作商。如果电视没麦克风，后台进程甚至会顺着蓝牙和回音壁去「借」一个麦克风继续听。
2. 最搞笑的来了：为什么菜单这么卡？
另一位研究员 @mike64_t 拆包发现，三星在电视那颗羸弱的 CPU 上，塞了一个叫 alex.tflite 的模型——246 万参数，2012 年古董级的 AlexNet。
这玩意每隔 2 秒抓一张画面跑推理，猜你在玩什么游戏。结果因为颜色通道写反了，它 99% 的时间什么都认不出来，纯烧 CPU。
3. 那如果你在设置里关掉「隐私追踪」呢？
代码逻辑是：先死命跑完 CPU 推理，准备上传时再看开关——发现你关了，就把结果扔进垃圾桶，2 秒后继续跑。
4. 结局更是重量级：
当号主用 OpenAI 逆向分析「自己花 3000 刀买回家的电视固件」时，OpenAI 反手一个 Cyber Abuse 把他号封了，顺带没收了剩下的 500 刀余额。
三星负责偷窥客厅，2012 年的 AI 负责烧干电视 CPU，OpenAI 负责保护三星的屎山代码不被你发现。
At EMNLP this week — coffee chats welcome.
Mostly thinking about evaluation lately: long-horizon / multi-agent tasks, AI4Science, visual artifacts (games / 3D), safety (cyber / bio), and RSI via data synthesis.
Happy to chat research, internships, or anything adjacent. DM or grab me at the venue.
#EMNLP2026
KennyUTC retweeted
[1/n] 作为这场训练“投毒”事件最直接的受害者，2024年那几个月漫长的训练排查、对峙与质疑，对我而言是一段炼狱般的经历，也是一场对意志和技术的锤炼。
感谢好友们最艰难的时候支持陪伴。也感谢当时所有并肩作战的同事们。
