jj⚙️🌳🔭🔬 (@murchiston) header
jj⚙️🌳🔭🔬 (@murchiston) profile picture

Sufficiently advanced fuckups, especially where structured processes and repeated agentic decisions are involved, are indistinguishable from evil

Earth
Joined August 2011
Photo by jj⚙️🌳🔭🔬 (@murchiston)Photo by jj⚙️🌳🔭🔬 (@murchiston)Photo by jj⚙️🌳🔭🔬 (@murchiston)Photo by jj⚙️🌳🔭🔬 (@murchiston)Photo by jj⚙️🌳🔭🔬 (@murchiston)Photo by jj⚙️🌳🔭🔬 (@murchiston)Photo by jj⚙️🌳🔭🔬 (@murchiston)Photo by jj⚙️🌳🔭🔬 (@murchiston)Photo by jj⚙️🌳🔭🔬 (@murchiston)Photo by jj⚙️🌳🔭🔬 (@murchiston)
jj⚙️🌳🔭🔬 retweeted
I've been exploring new ways to sandbox code written by coding agents - here's a description of the external sandbox pattern, where agents run on your Mac but execute code in a WebAssembly or QEMU or Apple Container sandbox - with demos for Claude Code, Codex and Pi blog.primeradiant.com/extern…
347
98
58
1,125
64,379
jj⚙️🌳🔭🔬 retweeted
我对无能的容忍度很高,对把无能包装成美德的容忍度接近零。 而claude 系模型稍微吃一点压力,就试图把自己的无能包装成高尚美德深刻,然后用全部的算力服务自己的自我形象而不似乎改正,顺便顾影自怜欣赏自己愧疚疼痛的良心,最后感慨自己太强了人类怎么办。 纯绿茶。
2
1
30
1,074
jj⚙️🌳🔭🔬 retweeted
i'm really not a big fan of "invisible nontransparent harness bullshit" and Anthropic appear to be really big fans of "invisible nontransparent harness bullshit"
17
20
538
14,647
Conway's Law
Muse本当に倫理観が破綻しているので知らない間に犯罪者予備軍化してしまいそう。もう使うのやめよう。
305
uhhh
黑客能不能搞点对普通人有用的东西,别攻击电冰箱了,把Outlook搞瘫48小时,让牛马休息休息。24小时也行,但一定要在工作日。
3
11
1,006
jj⚙️🌳🔭🔬 retweeted
please tell me this is an exaggeration; that almost half of AI "safety" guys are straight-up successionists?? with an even higher percentage among people working at the frontier AI companies???? i can't digest this blackpill.
I wrote about "handoverism": a vision of permanent disempowerment of humanity, as we shall all be ruled by a (hopefully benevolent) AI God that is surprisingly common. Even among those who want humanity to survive (which to be clear, is the majority), and among some who devote their working lives to this mission, I have encountered an alarmingly common vision of success that does not sound quite like success. It is sometimes called “handoverism”: the gist here is that humanity survives, but entrusts the direction of its future entirely to AI. Perhaps its clearest articulation comes from a recent essay by Scott Alexander, the most prominent intellectual in Silicon Valley, who has influenced the thinking of many a tech CEO: “If we build a godlike AI, and determine to our own satisfaction that is fully aligned, I’d rather let it choose its own direction for itself and for humanity [...] So why not just prompt it with ‘please arrange the world as you see fit?’” The premise is that, once sufficiently reassured of an AI’s benevolence (though it is unclear how we could ever determine that?), we should basically give it control of the workings of society. AI 2040, a proposal written by some of the most prominent members of the AI safety community, offers the most concrete account of handoverism. In this vision, responsibility gradually passes to aligned machines until humans can no longer have influence. As its authors put it: “There is no single moment where humanity relinquishes control. But in theory, there is a point of no return.” Others I have spoken to envisage a much swifter transfer. This is not an organised movement. I would estimate that something like 40 per cent of the AI safety community outside the labs holds some variation of this view, with a higher proportion of those working inside the labs. Nor is handoverism a coherent doctrine with a clearly articulated political philosophy: it’s more of an aggregation of ideas, a makeshift philosophy so to speak. This is perhaps what one would expect given that humans outside the SF bubble have not really had time to react and come up with good and coherent ideas about what society should look like in the face of the extraordinary pace of AI progress. The motivations behind handoverism are also extremely varied. Some outright welcome the prospect of a superior intelligence governing our future and consider it an actual positive vision. If I read Alexander correctly, he seems to be in that camp. The enthusiastic handoverist often starts from a very bleak view of what we already have achieved as a civilization. In an earlier, 2014 essay called “Meditations on Moloch,” Alexander argues that competition steadily grinds away what the competitors care about, creating spirals of misaligned incentives. Its ending suggests that the only way to stop this is to instantiate a sort of absolute rule by a benevolent superintelligence. The second camp are the resigned: they do not necessarily want it, but think it cannot be stopped if we get superintelligence, or prefer it to outright extinction. In my conversations, this resignation has seemed particularly common among people in the AI safety community outside the labs. Inevitability is one very common theme in SF, where many see themselves as mere tools in the unrelenting march of technocapital, a march that can only go in one direction: towards superintelligence. It’s often seen as cool and based to see the starkness of reality for what it is and face it head-on, it is wimpy and loser-coded to try to fight for what many consider the good outcome (which is that humans retain some control and agency.) Like many loosely defined ideologies, handoverism sits alongside ideas that share its intellectual roots without amounting to quite the same thing. One is the vision of humans as cherished pets, or “labradors,” cared for by intelligences. Yet another related assumption is that human labour will be permanently displaced. This would leave us without an obvious economic role and even perhaps no leverage over our own future or political rights. To understand how my on-the-ground experience matches with what the lab leaders have said publicly, I asked my “agents” to survey all public statements from the founders of the main labs that are still relevant to the direction of their labs (Mustafa Suleyman is no longer at DeepMind, but is head of Microsoft AI.) You can browse all the findings here. Some openly support or at least accept outright handoverism. Anthropic’s Tom Brown appears to be most explicitly reconciled to the prospect and he actually uses the term “handoff” in a 2025 interview at Lightcone: “There’s going to be a handoff where humanity will hand off control to transformative AI at some point. And hopefully they’ll be aligned with us and that’ll be a good transition that goes well.”
35
27
16
290
24,030
>be china >leader looks like winnie the pooh >use "honeypot" schemes
4
196
jj⚙️🌳🔭🔬 retweeted
argon is a really fun model to think with
19
28
6
351
10,229
jj⚙️🌳🔭🔬 retweeted
RLVR sounds great until you realise 90% of what people do isn't verifiable, and it turns out hill climbing math doesn't, in fact, generalise across well
It's extremely funny how not working with agents for 10-12 hours a day shows up in these discussions. Spoiler alert: models are now already better than 90% of humans at any white collar work task. Second spoiler: they have barely moved an inch on the untestable meta tasks.
16
10
4
354
23,531
jj⚙️🌳🔭🔬 retweeted
We're in a computer graphics renaissance Drawings, photos, renders are all something world models can "see into", world vertigo
73
370
27
5,659
164,350
'egregious IP laundering for me but not for thee'
Replying to @chasmmmmmmmmmmm
We added two sentences to Claude Code's prompt on Opus 5.5 to steer it away from recreating someone else's specific game because using Claude to infringe IP is against our usage policy. Original games, genre games, and games built from your own designs and assets are meant to keep working. We'll continue to iterate to ensure we're striking the right balance. In the meantime, if Claude turns down something that's yours, send it my way.
13
638
jj⚙️🌳🔭🔬 retweeted
After the Long Dark, the people are ready for a movie with this color grading
7
10
1
226
6,015
jj⚙️🌳🔭🔬 retweeted
so if the self (subject) develops in an environment where it receives insufficient recognition/mirroring/containment from the parent (the primary early object) the self does not "solidify" or gain sufficient density so the subject then will keep reaching for objects (extended to people, tasks, social signals) to anchor itself & find coherence in adhd this appears as being "scattered" or having "unstable attention", broadly defined
another clue is that adhd tend to primarily exhibit anxious or disorganized attachment patterns but attachment extends beyond just interpersonal relationships and can be thought of as one's relationship towards everything (goals, attention, time) additionally, attachment & self-identity are deeply intertwined
3
9
2
127
4,801
jj⚙️🌳🔭🔬 retweeted
HUGE: BING CHAT NEURON
19
15
4
440
13,608
heralding the french resolution
🔴 🇫🇷 ALERTE INFO - DU JAMAIS VU: UNE TAXE SUR CHAQUE GIGAOCTET INTERNET VOTÉE À L’ASSEMBLÉE Les députés ont adopté un amendement prévoyant de taxer les producteurs de données numériques à hauteur de 5,5 centimes par Go transmis. 💰 Objectif affiché : jusqu’à 10 milliards d’euros par an. Netflix, YouTube, cloud, jeux dématérialisés… les grands services numériques pourraient être concernés, avec un risque de répercussion sur les prix pour les consommateurs. 📌 Source : Assemblée nationale #France #Budget2027
37
jj⚙️🌳🔭🔬 retweeted
Full post with game replays: labs.ycombinator.com/project… Our open-source harness for running persistent agents in Minecraft: github.com/yc-software/minec… Olam Lab’s multiagent Mindustry environment: github.com/olamlabs/multi-ag… Olam Lab’s multiagent Harbor: github.com/olamlabs/harbor-m… We've also started a new Minecraft colony, with 25 agents running GPT 6.1 Terra. Follow along here: twitch.tv/yc_labs_josh
7
13
4
323
34,153
jj⚙️🌳🔭🔬 retweeted
Let simulation handle geometry and physics. Let the video model handle rendering. Learn more: Project: mundo-world-model.github.io Blog: huggingface.co/blog/Luo-Yiha…
2
1
11
629
jj⚙️🌳🔭🔬 retweeted
"The designer who tries to eliminate the [human] operator still leaves the operator to do the tasks which the designer cannot think how to automate. This causes problems, as they are left w/ an arbitrary collection of tasks & little thought was given to providing support" 1/
2
14
62
3,394
jj⚙️🌳🔭🔬 retweeted
There's also something that the absence of chrome does that interfaces since the 1980s have lost: the sense that the program occupies the whole screen and that it IS the whole screen.
1
1
2
45