"And if a sci-fi writer tried to write that story, the audience would just be confused, because why did that happen? Isn’t that the opposite of what the AI was trained for? But reality is allowed to be like that. And we are, fundamentally, predicting that the world will not turn out like a sci-fi novel. We’re predicting that AI’s preferences will turn out to be complicated and weird."