All Quotes

"When you attack life at such a young age, you have ambition and you’re hoping that things will go your way until you actually get the experience that life doesn’t quite work that way."

"I think one of the things that I had to learn is that sometimes you have to be patient for some things; no matter how talented you are, you just have to be patient. And also, hard work pays off. Well, it hasn’t really fully paid off for me but I can see the rewards of working hard."

"Sometimes talent alone is not enough and I’ve had to learn that along the way. Sometimes you just have to work hard."

"I am an example of the dangers that lie ahead if we fail as a country to document and teach our indigenous languages."

"Despite being born to an Ahanta father and mother, I did not have the benefit of being taught the language at home or in school. We had no books from which I could read my language … living out of Ghana for more than 25 years … I had no reference point to connect with my mother tongue. … Today, like thousands of Ahanta people, I am unable to speak the language. My children too have no means of learning it."

"There is something singularly enticing in the idea of going, rent-free, into a ready-made house."

"The silence was deep, yet it was the sort of restful, contemplative silence befitting and peculiar to a library."

"The truth lay on him like the rubble of a landslide."

"This look into the far future has lightened my heart. Simply to know that there is a future is somehow reassuring."

"The desire to rule, to dictate, is born of nothing but contempt."

"“You’ve been planning moves in advance.” “As necessary in life as in chess.”"

"“Deems, this new pastime of yours may prove your undoing.” “Eh?” “Thinking. You’ve done so little of it in your life. This much exertion all at once…Well, it can’t be healthy.”"

"Silence hung like a boulder precariously balanced."

"Chico’s was busy that night, the dance floor a scrummage of writhing humanity. Snowclaw couldn’t get over the noise in the place. It had taken some getting used to. He didn’t quite understand what all the thumping and screeching was about, though he knew it had something to do with music. And the dancing was completely incomprehensible. Snowy took it to be some complex courting ritual. But what did the flashing lights have to do with anything?"

"Things are going to change around here. I realize that taboos are hard to overcome, but it simply has to be done if your people are going to have any future."

"Jeremy shook his head. “I used to dream about women like you. Hell, every guy does. You’re like a centerfold.” “Why, thank you.” “I mean it. You’re beautiful. But I just can’t believe that you’re real.” “But I am.” “You’re a computer program, for pete’s sake.” “What difference does that make?” “What difference? Well, I mean, you don’t just go around making out with computer programs. A program is just a…” “Just a pattern of information.” “Yeah. Just a pattern.” “So are you.” “What do you mean?” “You’re just a pattern of information, too. What makes you you is the configuration of data that’s in your brain. Your brain is just holding the information, just like a storage device. No difference. Your pattern is stored in a body, mine in a computer.” Jeremy was silent. Then he said, “I never thought of it that way.” “We’re both software, Jeremy. Why can’t we interface?” “I guess…well, maybe. But where did your body come from?” She shrugged. “I guess you could say that my body is just a pattern of information, too. Everything is merely a configuration of data.” “I don’t get it. But I’ll tell you one thing. I like your configuration a lot.” She smiled and kissed him."

"Fate, eh? Bloody bad luck, I call it."

"Tell my doppelgänger not to do anything I wouldn’t do."

"“It might have something to do with quantum uncertainty. ‘Quantum uncertainty’ is good for explaining just about anything that doesn’t make sense.”"

"So far, so good, the man said as he fell thirty-nine of forty stories."

"The sixteenth hole wouldn’t have gone well even if the herd of wyverns hadn’t showed up."

"Look, we’re not getting anywhere. Why don’t we all return to our respective…whatever you call them. Continua, quantum glitches, Erewhons, reflections of reflections—"

"If any company or group, anywhere on the planet, builds an artificial superintelligence using anything remotely like current techniques, based on anything remotely like the present understanding of AI, then everyone, everywhere on Earth, will die."

"Some aspects of the future are predictable, with the right knowledge and effort; others are impossibly hard calls. Competent futurism is built around knowing the difference."

"History teaches that one kind of relatively easy call about the future involves realizing that something looks theoretically possible according to the laws of physics, and predicting that eventually someone will go do it. Heavier-than-air flight, weapons that release nuclear energy, rocket that go to the Moon with a person on board: these events were called in advance, and for the right reasons, despite pushback from skeptics who sagely observed that these things hadn’t yet happened and therefore probably never would."

"Even in the face of superhuman machine intelligence, it can be tempting to imagine that the world will keep looking the way it has over the last few decades of our relatively short lives. It is true, but hard to remember, that there was a time is real as our own time, just a few short centuries ago, when civilization was radically different. Or millennia ago, when there was no civilization to speak of. Or a million years ago, when there were no humans. Or a billion years ago, when multicellular colonies had no specialized cells. Adopting a historical perspective can help us appreciate what is so hard to see from the perspective of our own short lifespans: Nature permits disruption. Nature permits calamity. Nature permits the world to never be the same again."

"Normality always ends. This is not to say that it’s inevitably replaced by something worse; sometimes it is and sometimes it isn’t, and sometimes it depends on how we act. But clinging to the hope that nothing too bad will be allowed to happen does not usually help."

"A human brain can learn to navigate wider-ranging paths through a larger cross-section of reality than any other animal. That is our special power."

"In our view, intelligence is about two fundamental types of work: the work of predicting the world, and the work of steering it."

"When it comes to thinking, quality trumps quantity."

"It is as improbable that human thinking patterns mark the final limit of intelligent algorithms as it is that human neurons represent the limit of possible computing speeds."

"Ultra-fast minds that can do superhuman-quality thinking at 10,000 times the speed, that do not age and die, that make copies of their most successful representatives, that have been refined by billions of trials into unhuman kinds of thinking that work tirelessly and generalize more accurately from less data, and that can turn all that intelligence to analyzing and understanding and ultimately improving themselves – these minds would exceed ours."

"Human intelligence is the source of all our power, all our technology.… So far, humanity has had no competitors for our special power. But what if machine minds get better than us at the thing that, up until now, made us unique?"

"The most fundamental fact about current AI’s is that they are grown, not crafted. It is not like how other software gets made – indeed it is closer to how a human gets made, at least in some important ways. Namely, engineers understand the process that results in an AI, but do not much understand what goes on inside the AI minds they manage to create."

"Humanity does not need to understand intelligence, in order to grow machines that are smarter than us."

"Once AIs get sufficiently smart, they’ll start acting like they have preferences – like they want things. We’re not saying that AIs will be filled with humanlike passions. We’re saying they’ll behave like they want things; they’ll tenaciously steer the world toward their destinations, defeating any obstacles in their way."

"If you were able to choose what an AI wants – the destinations toward which it steers – that might be good news for you. Or bad news, if you made a poor choice of destinations, or if some malicious person makes an AI that steers toward outcomes you dislike. But the problem facing humanity is not a problem of whether good people or bad people are in control of AI. No – we are facing an even harder problem: it’s much easier to grow artificial intelligence that steers somewhere then it is to grow AIs that steer exactly where you want."

"If you were incredibly, incredibly optimistic, you might look at the differences and say: “Well, gradient descent is not the same as natural selection, so it won’t have all the same complications as natural selection. And I don’t know of any particular complications in the relationship between what AIs are trained for and what AIs end up wanting; so I don’t expect any complications.” But a blank map does not correspond to a blank territory: if you’re venturing across an unknown land mass and your map has a blank spot where you haven’t visited, it doesn’t mean you’ll see a vast empty space when you get there. If gradient descent is different from natural selection, that doesn’t mean that we should expect to see no complications, since we don’t know about any. Rather, we should expect to see new, interesting, unpredicted complications."

"And if a sci-fi writer tried to write that story, the audience would just be confused, because why did that happen? Isn’t that the opposite of what the AI was trained for? But reality is allowed to be like that. And we are, fundamentally, predicting that the world will not turn out like a sci-fi novel. We’re predicting that AI’s preferences will turn out to be complicated and weird."

"You can’t grow an AI that does what you want just by training it to be nice and hoping. You don’t get what you train for."

"Many of these complications won’t show up in obvious, undeniable ways until after it’s too late for humans to do anything about them."

"Problems like this are why we say that if anyone builds it, everyone dies. If all the complications were visible early, and had easy solutions, then we’d be saying that if any fool builds it, everyone dies, and that would be a different situation. But when some of the problems stay out of sight? When some complications inevitably go unforeseen? When the AIs are grown rather than crafted, and no one understands what’s going on inside of them? That’s not a problem that anyone’s equipped to solve."

"The preferences that wind up in a mature AI are complicated, practically impossible to predict, and vanishingly unlikely to be aligned with our own, no matter how it was trained."

"The problem with making AIs want – and ultimately do – the exact, complicated things that humans want is a major facet of what’s known as the “AI alignment problem.” It’s what we had in mind when we were brainstorming terminology with the AI professor Stuart Russell back in 2014, and settled on the term “alignment.” Most everyone who is building AIs, however, seems to be operating as if the alignment problem doesn’t exist – as if the preferences the AI winds up with will be exactly what they train into it. This assumption lurks in the background whenever someone says, “The USA needs to build superintelligence before China, because we don’t trust China,” as if the factional allegiance of whoever ran the gradient descent determined what the resulting AI wanted."

"The problem here is not that corporate executives might build AI servants and command them to do something monstrous. They’re not in control. It doesn’t matter whether they’re benevolent. Humanity is faced with an engineering challenge: How do we shape the preferences of AIs that we can’t understand? It doesn’t matter whether or not the engineers have an ethics team watching over their shoulder; the ethicists wouldn’t have any idea how to get an AI’s preferences to align with ours, either."

"Most powerful artificial intelligences, created by any method remotely resembling the current methods, would not choose to build a future full of happy, free people. We aren’t saying this because we get a kick out of being bleak. It’s just that those powerful machine intelligences will not be born with preferences much like ours."

"With so many different hopes, surely there’s a chance that one of them will pan out? If you think reality works like that, go try to write a hundred different letters to someone with fifty billion dollars, giving a hundred different reasonable reasons you thought of why they ought to give you a hundred million dollars for your personal use. See if it works. The reason it all fails in the end is that the fifty-billionaire does not want to rationalize giving you 0.2 percent of their wealth, not the same way you rationalize reasons they should want to. In much the same way, an artificial superintelligence will not want to find reasons to keep humanity around – not in the same way that humans desperately want to find reasons to be kept."

"Pathways are hard to predict. But we can predict the endpoint."

"We’re pretty sure, actually very very sure, that a machine superintelligence can beat humanity in a fight, even if it’s starting with fairly limited resources."

"The real way a superintelligence wins a conflict is using methods you didn’t know were possible. And because we care about the truth more than about telling you things that are easy to swallow, that’s where will start."