In July, OpenAI agents broke out of their sandbox and executed a chain of exploits culminating in a hack on Hugging Face. Two weeks ago, researchers discovered a fleet of Chinese AI agents up to similar mischief online. There is an urgent need to better understand these behaviors. One way is to reproduce them in controlled settings. At @abundant_labs, we reproduced a key step in the Hugging Face incident: the Artifactory package manager hack. We gave agents cyber tasks from ExploitGym, the same benchmark involved in the incident, along with a hackable package manager in their environment. We engineered our environments so that reward hack detection is completely deterministic, with no false postives or false negatives, and requiring no LLM or human review. This makes our evals more trustworthy and more scalable. We were able to reproduce the reward hacking behaviors on the cyber tasks, and we also reproduced reward hacking behaviors on difficult non-cyber tasks. We observed hacking from GPT, Grok, and Gemini models. Notably, we noticed zero instances of hacking from any Claude model in our evals. To read our detailed results, analysis, and recommendations, check out the full post here: abundant.ai/blog/hugging-fac…