btw, i think it's *totally possible* that we're all just wrong about near-term x-risk. like through a combination of selection effects, drinking the koolaid and mutually reinforcing each other's views, we've worked ourselves into a panic over an implausible scenario (read to end) like maybe there is a huge silent majority of AI experts who think our concerns are totally silly! it's impossible to determine whether this is the case, because people who are unworried about x-risk usually don't discuss it maybe alignment is trivially easy. maybe it turns out not to be a real problem. maybe we really are just building tools and not creatures. maybe ASI isn't even possible due to some real-world constraints we don't know about but i think my current level of alarm RE AI x-risk is totally warranted, even while believing that the above is plausible! if in 20 years none of this has played out, I don't think i will have been wrong to worry. I'm increasingly frustrated by the inability of some on the skeptic side to exhibit this same humility. "what if we're wrong?" is a far more important q for them to ask than it is for us! this rant is inspired by an event on AI governance I went to last night (which I will not name). the panellists were just so confidently, egregiously dismissive of x-risk that I declined to stay for the free dinner afterwards so no one would see the smoke coming out of my ears these were important, credible ppl with real expertise! while they were describing the silly little x-risk panic of 2023 as an episode of misguided alarmism that scared so many poor laypeople, I could feel the colour rising in my cheeks, embarrassed to be accused of credulity what angers me here is not that these experts do not believe in existential risk. it's that they unfaithfully represent the state of the discourse. they fail to acknowledge their many peers who take it seriously. they ignore the total lack of consensus in the field people communicating to the public about x-risk owe us honesty. we deserve to know that there's no pilot flying this plane. If you're an expert who is (somehow) sure that birthing an intelligence greater than our own poses no risks, caveat that this is far from a settled debate also, trust your instincts. don't let anyone make you feel silly for worrying about AI x-risk. the arguments for it *are* as straightforward and intuitive as they seem. even if they turn out to be wrong, you won't have overreacted