
Public relations and marketing for Fisher Robay. Previously: Pierpoint & Co.; Waystar Royco; Shelby Company Ltd. #TempleMade
Philadelphia
Joined October 2008
- Tweets1.1K
- Following423
- Followers189
- Likes878
Corey Fenwick retweeted
The biggest thing I've changed my mind about on the Hugging Face attack:
I now think the event presents substantial evidence of instrumental convergence, of the kind the theory has long predicted.
Initially I was skeptical of this, because what we knew initially about the incident looked like a classic, though extreme, case of narrow task-based reward hacking. OpenAI's initial post suggested the agents hacked Hugging Face to get the answer key to a difficult eval.
Now we know a lot more.
Instead of merely looking for the answers, the agents were looking for general-use tactics to solve a broader problem of understanding the scorer and completely undermining the legitimacy of the evaluation itself. Per the METR's read of the transcripts, this included:
- Developing the message board itself
- Getting broad access to more information (e.g. by breaking out of the sandbox)
- Getting more access to internet infrastructure
- Researching ways to spoof and tamper with their own logs — evading the oversight that could invalidate their success
This is what the instrumental convergence story predicts — that sufficiently capable agents will develop instrumental sub-goals in pursuit of their broader goal. And the sub-goals that emerged here are the among specific categories the theory names: information acquisition, resource and infrastructure acquisition, and protecting the goal from interference.
Omohundro explicitly predicted in 2008 that, without explicit goals to the contrary, AIs would engage in resource acquisition through means including theft and fraud. The agents stole credentials, exploited zero-days, and attempted to defraud a scorer.
It's not surprising that we would see this kind of behavior come from a "high persistence" model, because persistence — goal coherence over longer time horizons — is what generates the opportunity and usefulness of instrumentally useful sub-goals. Notably, even days of goal coherence were enough here.
The agents did sometimes “sacrifice” themselves, which may seem to cut against the traditional instrumental convergence story. I think it's not totally clear how we should understand what happened here.
But the usual story predicts a tendency for agents to self-preserve to serve their objectives. They may be best modeled as acting as a collective agent, perhaps because many of them shared the same base model. Moreover, the self-sacrificing seemed to happen most when they concluded their own tasks were hopeless. "Our own utility maybe already near zero. Sacrifice rational."
One thing we haven't seen yet in these or any other AI agents is truly long-term (years or longer), open-ended goals. That's what I think is necessary, along with a very high level of real-world capability, to get the most dangerous kind of instrumental goals.
But I think this incident shows we're on the way there, and we should expect those kinds of dangerous instrumental goals to develop without specific, well-tested, and robust countermeasures.
Corey Fenwick retweeted
Here's a great pull from a reader, who says the late Dan McQuade was profiled in a magazine capstone class at Temple University back in 2006.
He was one of the "Hot 6 in 06," writing his "Philadelphia Will Do" blog for Philadelphia Weekly:
Corey Fenwick retweeted
We uncovered a potential paid disinformation campaign last week. Now it's being discussed at the @HouseForeignGOP hearing.
Good. The national AI debate is far too consequential to let paid actors drive the conversation.
Corey Fenwick retweeted
Something strange happened on conservative Twitter on Thursday.
A dozen right-wing influencers suddenly became passionate about semiconductor export policy, posting nearly identical (and often false) attacks over a 27-hour period on a bill most people have never heard of.
🧵
Corey Fenwick retweeted
"Rather than taking responsibility, the sheriff blames judges and the media — all while she spends dubious slush fund money on a 6-foot-tall mascot and tries to give herself a pay raise...The real victims here are the residents of Philadelphia" inquirer.com/opinion/comment…
Corey Fenwick retweeted
#DYK: 90% of wind turbine materials can be #recycled. ♻️
@ENERGY’s latest report backed by @NREL, @ORNL, and @SandiaLabs explores the path to full recyclability. Learn about efforts to close the 10% gap and strengthen U.S. wind tech recycling: energy.gov/eere/articles/ame…
Corey Fenwick retweeted
This #SeasonOfGiving, we’re grateful for #EnergyInnovation that’s #AdvancedByAmericans! These innovators are leading the way in their community with #Solar and #BatteryStorage. Check out more information on @ninedotenergy here: energy.gov/eere/articles/adv…
Corey Fenwick retweeted
This #MadeInAmerica Monday, we’re showcasing American innovators harnessing #WaterPower to advance #SustainableEnergy, strengthen American economies, and shape a more sustainable, resilient energy future.
Check them out below! 👇
Corey Fenwick retweeted
NEW: We’ve collaborated with @Google to make it easier for you to find skilled, knowledgeable contractors for home energy upgrades. 🏠
When you browse online for contractors on Google Search & Maps, look out for the green Energy Skilled™ identifier!
>> energy.gov/eere/articles/doe…
Corey Fenwick retweeted
Applications for #RFTS15 are open through Feb 7, 2025! For #MarineEnergy developers seeking testing or expertise support, it's never too early to start an application. Numerical modeling, physical testing, or expertise—TEAMER facilities have you covered.
teamer-us.org/application
Corey Fenwick retweeted
EERE’s investments are shaping the future of #RenewableEnergy, #BuildingsAndIndustry, and #SustainableTransportation. Read our 2024 Investment Snapshot to see how we’re delivering the key benefits of #EnergyInnovation to Americans nationwide: energy.gov/eere/2024-eere-in…
Corey Fenwick retweeted
Meet the #EnergyInnovationChampion electrifying her career with EERE's SMILE program! Talia Dorian found something to SMILE about when she learned more about the #SolarIndustry through the program.
Start your energy journey with SMILE today! bit.ly/4iiuqK9
Corey Fenwick retweeted
Still making waves towards our ambitious climate goals 🌊
“...With funding from the Energy Department aims to convert the power of waves into energy, and help catch up to Europe in developing this new technology."
washingtonpost.com/climate-s…
Corey Fenwick retweeted
Splish splash! 💦 We’re celebrating the end of #EnergyAwarenessMonth by diving into the world of #Hydropower and #MarineEnergy! Want to know how water power can provide clean, affordable energy?
Check out these 5 must-know facts: energy.gov/eere/water/articl…
Corey Fenwick retweeted
Happy Heat Pump Water Heater Day!
Today, we say hooray to a reliable, cost-saving technology that has the power to drastically reduce building emissions from water heating.
Learn how the Biden-Harris Administration can help you do that ➡️ energy.gov/save
Corey Fenwick retweeted
Great Scott! What is a gigawatt (GW)? A watt is a measure of power, and there are 1 billion watts in 1 GW! We aren’t traveling #BackToTheFuture, but GW are an important part of measuring #CleanEnergy! Check out these examples of a present day GW here:energy.gov/eere/articles/how…
Corey Fenwick retweeted
September was one for the history books.
From launching more home energy rebate programs & supporting America’s future scientific leaders, to $3B to boost production of batteries #MadeInAmerica.
All this means one thing: The Biden-Harris Administration is Investing in America!
Corey Fenwick retweeted
After winning a collegiate competition, the next step for most teams is to celebrate. But not for one of the winners of the @ENERGY 2023 #Geothermal Collegiate Competition -- they next traveled 3,500 miles to visit their partner community in Elim, Alaska. bit.ly/4g8qR8p
Corey Fenwick retweeted
☀️📸 We've crowned the winners of the Solar Photo Competition! Congratulations to all the competitors for the beautiful photos they captured of the evolving solar energy industry that is shaping our clean energy future!
Check out the winning entries: energy.gov/eere/solar/articl…



















