⿻ Audrey Tang 唐鳳

Q&A: @audreyt & @VitalikButerin Ethereum’s priorities for the next decade – first priority is surviving. institutions appreciate Ethereum not for speed, but for the fact the chain has never gone down. if Ethereum is not secure, it can’t make anyone else secure. if it’s not private, it can’t help anyone else be private. if it’s not open source, nothing built on top is open source – stop doing: chasing 5% improvements when AI is turning everything over every year. chase the 5x instead. “helping people get 4.8% APY instead of 4.2% APY is fundamentally not interesting”. the applications that matter reduce the chance of a negative 100% APY. make simple and basic things good rather than hyper-optimising everything fragmentation – core R&D participation still has elements of being a social game. AI competitions may help: everyone who contributes is technically a core dev. anything that can be measured gets accelerated through AI; the things that can’t become the hard problem, but at least a smaller one – privacy is the clearest example of taste-level fragmentation: data confidentiality, anonymity, metadata privacy, short-term privacy (front-running resistance), payment privacy, DeFi privacy... all different trade-offs with provable upper limits. sometimes different positions on the impossibility triangle are legitimate; sometimes a bigger solution turns the tribal warfare into a slider (even presenting the slider as common knowledge is progress, right now it feels zero-sum) AI agents managing community treasuries –a special case of the alignment problem, or “the ultimate problem facing humanity”. expects not one answer but a hundred answers each solving 1% of the problem – any agent governing community assets sits in a sandbox, even when it has the keys: who picks the AI, who controls training, who decides on upgrades, who defines what counts as a big decision, what’s the process for feedback. those sandbox rules still have to be decided by people – alignment by process: agentic systems earn trust by trusting the community more, not by a single designer setting rules from above privacy vs transparency – it’s about power asymmetry. concentrated power owes more transparency; lesser power is owed more privacy. the North Star is de-totalisation – in practice, lines don’t get drawn by philosophers, they get drawn when two factions fight long enough, get tired, and need a peace settlement they can live with. that’s how the Magna Carta and the Peace of Westphalia happened – influence it by building practical solutions that give more of the good kind of transparency and more of the good kind of privacy at the same time – “be the agent you want to see in the world” agents onchain – audience question: if everyone can see how my agent works, scammers can too - does openness actually make it safer? answer: “I don’t think we want to put our agents on-chain” P(doom) – Audrey: “my P(doom) is NaN” - current model interpretability can’t read the neurons of very large models. visibility is zero; zero divided by zero is Not a Number – on cyber apocalypse: it’s already happening. “assume breach” used to be a slogan; now anything you collect is probably already breached. the real apocalypse is the day all that leaked PII comes back to bite everyone AI meets Ethereum (Vitalik) – most excited: speed-running cryptography. the zero-knowledge proof singularity - a single ZKP verifying everything. indistinguishability obfuscation was always “30 years away”; AI is accelerating things he thought wouldn’t arrive for a quarter century – most worried: “it’s like the dog-in-a-fire meme. we are the dog. the fire is creating diamonds that could go into chips and make everything awesome, but it’s also burning the house, and we have to take care of the house” which phone does Vitalik have? does he use hot or cold wallets? which fridge does he have in his house? what does he think of India? watch the stream of @EFDevcon to find out! teapawt.net/EFDevcon/status/211746…

wow, this was an extremely insightful talk! @audreyt on Civic AI for Self-Governing Communities privacy vs transparency: a power asymmetry rule – the more power you hold over others, the more transparency you owe them. if you hold no power over anyone, you should not be made transparent to anyone – for any piece of data, ask one question: does it record who someone is (allowing someone to dox them), or does it record the fact that someone holds power over others? the first must be protected, the second must be made transparent – elections already work this way: the count is public and verifiable, but individual votes stay private. the Ethereum ecosystem has voting systems built on the same principle, where proofs show only the aggregate – who draws the line? the people affected write the rules together, and the rule set is public. in AI systems those rules are called constitutions, model specs, charters. Tang has been advocating for all frontier labs to publish theirs in the public domain so communities can fork them – example: when Claude’s constitution was being drafted, they took the previous version, sent it to thousands of random US residents who deliberated using Polis. citizens quickly found loopholes, e.g. the spec assumed everyone walks, forgetting people in wheelchairs. the delta became a testing suite (eval), proving the new version respects disability rights without losing capability elsewhere selective disclosure as the default – most systems today ask for a whole document when they only need one fact. Taiwan’s wallet gov system embeds selective disclosure into the national identity system, proving you’re of legal age without revealing birthday or address – critical: these systems need to be more convenient than the alternative. like switching from passwords to passkeys: if the new way is harder, nobody flips – the Ethereum ecosystem’s Anon Aadhaar reuses India’s Aadhaar system with selective disclosure on top; influenced Japan’s My Number equivalent – with AI-driven vulnerability discovery accelerating, mass-collected personal data will quickly be seen as toxic waste. zero-knowledge and selective disclosure systems that don’t create more toxic waste will win the political window when digital ministers have to tell parliaments “never again” AI agents managing community treasuries: soft rules vs hard rules – Freysa experiment: an agent guarding a prize pool with instructions to never send the money. 481 messages failed, then message 482 convinced it that the “send money” tool was actually about “receiving money” via a small donation, and it sent the entire $47K pool – lesson: never treat autoregressive systems as following hard rules. any rule they claim to follow is a soft rule. soft rules need hard guardrails around them – Tang’s CivicAI framework separates instructions to models (which can only ask and advise, and have thousands of ways of being broken) from invariants that are formally verified - either by a specialised smaller model or by software like Bend2, which translates invariants into GPU-parallelisable, proof-carrying code – chain-of-thought is not trustworthy either: research shows models can reason internally that something is important to hide from the user. reading activations is a research direction, but for now, the practical answer is to define the action space and write invariants for the hard limits human in the loop → agents in the loop of humanity – individual humans pulled into the loop of an AI system increasingly feel like hamsters on an ever-faster wheel, just pressing enter – flip it and bring agents into existing loops of humanity: shared spaces like group chats, Slack, Discord. the community already has a code of conduct and a preferred pace. synthetic intimacy (“your ideas are so load-bearing!”) gets checked by the group - the more an agent tries to flatter an individual, the more cringe it looks to everyone else in the channel code, law, or norms? – all three, because each does a different job. code holds limits that can be specified formally, but there are always exceptions. people holding norms judge the exceptions. law summons the responsible person when norms fail – Taiwan’s anti-fraud law in practice: citizens flag scams through a public crowdsourced system. after official verification, platforms have 24 hours to take down the ads. if they don’t and someone loses money, the platform is liable for the full amount. the individual flagging stays private; the platform’s response is radically transparent relational agents and breakups – if two people share a model for their relationship and then agree to leave, what happens to the agent? – answer: agree to the terms of ending before the model even starts building memory. each person’s private notes stay on their own device. shared knowledge lives in a common pool, but it only keeps links (UUIDs), not actual private details. either side can disconnect and invalidate links at any time – technically: attribution-based control. exercising the right to be forgotten in a pre-trained model is nearly impossible. but if you never aggregate in the first place (each person and small community trains their own pod, and a coordinator model (like OpenRouter or Sakana) just routes queries to relevant pods), then breaking up means taking your pod with you and your fair share of contributions. “nobody will feel that their digital twin is still being captured and enslaved in a community model” – this also solves copyright attribution: if you don’t aggregate but always route, you can calculate on every query how much each pod contributed de-totalisation in practice – from the EF mandate: no organisation, no system, no moral order should have total dominance over any individual’s life. “I like to think of myself as a good enough ancestor. Not one who designs out all the bugs so the next generation can only follow our code, but one who leaves them a wider canvas with more materials for reuse” – practical habits: never use just one chat model. Tang uses a system prompt that forces HTML output with no first-person pronouns so the model can never pretend to be a friend. uses multi-agent harnesses (currently OMO) with coordinator, planner, verifiers, and workers, each must convince adversarially trained models that it’s fulfilling the actual ask – key finding: if you assign all roles to models from the same pre-training company, they collude easily. if you mix genuinely different pre-training bases (e.g. Gemini vs. Claude), they are adversarial against each other and don’t repeat each other’s blind spots – best practice: after using multi-model harnesses for a while, train your own local post-trained model photo courtesy of @EFDevcon