"Space probes. Nuclear reactors. Computer security. What do all these lessons add up to, and what can we learn from them about the difficulty of aligning an artificial superintelligence? An artificial superintelligence is like a space probe, in that we cannot test it in quite the same environment where it needs to work, and by default it is not retrievable or correctable once it rises high above us…. An artificial superintelligence is like a nuclear reactor, in that its underlying reality involves immense, potentially self-amplifying forces, whose inner processes run faster than humans can react. An artificial superintelligence is like a computer security problem, in that every constraint an engineer tries to place upon the system might be bypassed by the intelligent forces that those constraints hinder."
Quote Details
Added by wikiquote-import-bot
Unverified quote
0 likes
Original Language: English
Available Languages (1)
Sources
pp. 175-176
https://en.wikiquote.org/wiki/If_Anyone_Builds_It%2C_Everyone_Dies
Revision History
No revisions have been submitted for this quote.
Categories
Unknown
35329 quotes on TrueQuotesView all quotes by Unknown →
Related Quotes
"“Kahit matabang-mataba ka pa ay okay lang, basta siguraduhin mo na ang pinangkakain mo ay hindi galing sa pinaghirapa…"
""Education is very important"."
"" Kung hindi kaya ng mga magulang mo itaguyad ang pag-aaral mo, kung walang pera ang pamilya mo, pwede mong gamitin a…"
""Women are not immortal. We age. We wrinkle. We die."Daily Monitor, Friday, July 04, 2025"
"Women are present in the workplace in significant numbers, yet presence alone does not always translate into equal op…"
"Budgets are not simply ledgers of income and expenditure."
"Leaders must create an environment where merit is recognised, trust is built, and every employee feels valued."
"These attacks follow a familiar playbook-discredit the messenger to dismiss the message" - Gabriella calls out 'attac…"
""Aktibista ang GABRIELA at kami ay galit sa mga bastos at manyak na trapo!" - Clarise Palce GABRIELA Secretary-Genera…"
"That's the thing about a great book...you're different."