Manuel Mueller-Frank

I found this long post from OpenAIs chief scientist a bit bonkers. Outstanding, but bonkers. Outstanding because you see someone smart and honest reason carefully through a very, very hard problem. Bonkers because of what follows from that reasoning. 1. We do not understand alignment ("I believe that no lab has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer. I expect and hope for voluntary slowdowns to become commonplace until shared safety bars are established.") You want to teach machines how to love? Anyone has any idea of how to teach anyone how to love? 2. We are proceeding by making it up as we go ("The study of deep learning-based AI is largely an experimental science. We put a lot of effort⁠ into building principled algorithms and making testable predictions, but fundamentally, our large-scale training runs are experiments, and we are sometimes surprised by their results. Moreover, as the systems become more capable, the results become harder to interpret.") 3. It is getting harder to figure out what the machines are doing and thinking ("The AI is becoming better at reasoning about and manipulating its own reasoning process. With improved pretraining performance, we also see the models become much smarter even without using verbalized reasoning at all.") 4. Hence lets go forward fast into the unknown ("The strongest argument I see for continuing to train much smarter models quickly is the need to build defensive systems against the dangers posed by other AI.")