How was the book?

Read this book if you want to fully understand the concept of AI and superintelligence.

My analysis of Nick Bostrom’s ”Superintelligence” will be a lenghty one and more detailed than ever. But there is a good reason for that. The book is very philosophical and in the same time very fundamental on explaining the basics of AI. Bostrom uses a lot of space to analyze the potential threats that the superintelligence concept consists, but I won’t bring those topics up more than needed. I’ll start the analysis by pointing out three fundamental topics of AI and superintelligence. The topics are – laws of robotics, the value problem and components of AI.

About the threats of superintelligence Bostrom quotes Isaac Asimov’s “three laws of robotics” concept:

I) A robot may not injure a human being or, through inaction, allow a human being to come to harm.

II) A robot must obey any orders given to it by human beings, except where such orders would conflict with the First Law.

III) A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.

Second Bostrom points out solutions to the value-specification problem for AI and superintelligence:

I) “Encapsulate moral growth”

II) “Avoid hijacking the destiny of humankind”

III) “Avoid creating a motive for modern-day humans to fight over the initial dynamic”

IV) “Keep humankind ultimately in charge of its own destiny”

After these decisions humans needs to decide what kind of AI to build – genie, oracle, sovereign or tool-AI (I’ll refer to these later in the analysis), and then decide the components to upload into these systems:

”I) Goal content component. What objective should the AI pursue?  

II) Decision theory component. Should the AI use causal decision theory, evidential decision theory, updateless decision theory, or something else?

III) Epistemology component. What should the AI’s prior probability function be, and what other explicit or implicit assumptions about the world should it make? What theory of anthropics should it use?

IV) Ratification component. Should the AI’s plans be subjected to human review before being put into effect? If so, what is the protocol for that review process?”

The content of the book is heavy to read, but inorder to understand the fundamentals of AI – you should master these topics.

What are the key learnings of the book? 

From human intelligence to superintelligence

What if the future AI will have an IQ of 6455? “As soon as it works, no one calls it AI anymore.” (John McCarthy). Most probably we cannot even imagine the world beyond AI or superintelligence.

Bostrom’s definition of a superintelligence is that it is like ”any intellect that greatly exceeds the cognitive performance of humans in virtually all domains of interest.” Currently even the most advanced AI system is below the human baseline. Bostrom predicts that ”may be reasonable to believe that human-level machine intelligence has a fairly sizeable chance of being developed by mid-century.” So we will have to wait at least 30 years before the start of the superintelligence era – maybe even 100 years. But because superintelligence is something that exceeds the human level of intelligence, we first have to develop the AI that can match human intelligence.

”When they expect “human-level machine intelligence” (HLMI) to be developed? The combined sample gave the following (median) estimate: 10% probability of HLMI by 2022, 50% probability by 2040, and 90% probability by 2075.”

How long from human level to superintelligence?

                                              TOP100     Combined

Within 2 years after HLMI    5%               10%

Within 30 years after HLMI  50%             75%

First and foremost ”if and when a takeoff occurs, it will likely be explosive.” The AI experience might resemble the first time you used Internet or even a mobile phone.

The path to superintelligence

”The principal reason for humanity’s dominant position on Earth is that our brains have a slightly expanded set of faculties compared with other animals. Our greater intelligence lets us transmit culture more efficiently, with the result that knowledge and technology accumulates from one generation to the next.” So in that sense the AI or superintelligence projects are aligned with our way of living and developing our culture.

A AI project is a large scale software development program ”with it’s advantages and disadvantages…. Large scale software projects might offer a closest analogy with AI projects, but it is harder to give crisp examples of typical lags because software is usually rolled out in incremental installments and the functionalities of competing systems are often not directly comparable.”

Compared to a typical software development program there will be also the benefits such as:

I) Editability.

II) Duplicability.

III) Goal coordination.

IV) Memory sharing.

V) New modules, modalities, and algorithms.

”The AI path is more difficult to assess. Perhaps it would require a very large research program; perhaps it could be done by a small group. A lone hacker scenario cannot be excluded either. Building a seed AI might require insights and algorithms developed over many decades by the scientific community around the world.” Just to give a perspective we should remember that the Manhattan Project employed 130 000 people although the great majority of people employed where blue-collar workers.

But then again. Take a look at Google. Google started as a two man project. ”It is possible that the last critical breakthrough idea might come from a single individual or a small group that succeeds in putting everything together.” This is a possibility for countries such as Finland. Even the international collaboration is a possibility for Finland and especially within the EU.

Cognitive superpowers

The world population of robots exceeds 10 million. In the ABB Finland every fifth worker are robots. AI-tools can already outperforms human intelligence in many domains such as backgammon, chess, checkers, scrabble, poker, bridge etc. The list could go on, but the main message is that bits and pieces are already here. Let’s take a closer look at important milestones for AI-tools by the year 2015:

·      Face recognition has improved.

·      Machine translation remains imperfect but is good enough for many applications.

·      Modern speech recognitionis sufficiently accurate for practical use

·      ”The Google search engine is, arguably, the greatest AI system that has yet been built. Automated stock-trading systems are widely used by major investing houses.”

Lessons from previous works:

·      Complications

o  One is the reminder that interactions between individually simple components (such as the sell algorithm and the high-frequency algorithmic trading programs) can produce complicated and unexpected effects.

·      The algorithm just does what it does

o  Another lesson is that smart professionals might give an instruction to a program based on a sensible-seeming and normally sound assumption (e.g. that trading volume is a good measure of market liquidity) and that this can produce catastrophic results when the program continues to act on the instruction with iron-clad logical consistency even in the unanticipated situation where the assumption turns out to be invalid.

·      Automation

o  Automation contributed to the incident, it also contributed to its resolution. The pre-programmed stop order logic, which suspended trading when prices moved too far out of whack, was set to execute automatically because it had been correctly anticipated that the triggering events could happen on a timescale too swift for humans to respond.

”The computer scientist Donald Knuth was struck that “AI has by now succeeded in doing essentially everything that requires ‘thinking’ but has failed to do most of what people and animals do ‘without thinking’—that, somehow, is much harder!””

Paths to superintelligence (direct quotes from the book)

How could and should we create superintelligence? Let us examine some possible paths. It now seems clear that ”a capacity to learn would be an integral feature of the core design of a system intended to attain general intelligence. The same holds for the ability to deal effectively with uncertainty and probabilistic information.” Here are presented Bostrom’s five different paths to superintelligence.

I) Artificial intelligence

Alan Turing’s notion of a “child machine,” which he wrote about in 1950: Instead of trying to produce a programme to simulate the adult mind, why not rather try to produce one which simulates the child’s? If this were then subjected to an appropriate course of education one would obtain the adult brain. Time needed to produce brain like superintelligence? Even a century of continued Moore’s law would not be enough to close this gap.

II) Whole brain emulation

A human whole brain emulation might be available around mid-century.

III) Biological cognition

A third path to greater-than-current-human intelligence is to enhance the functioning of biological brains. Many generations would be required to produce substantial results. Our individual cognitive capacities can be strengthened in various ways, including by such traditional methods as education and training. A world that had a large population of such individuals might (if it had the culture, education, communications infrastructure, etc., to match) constitute a collective superintelligence.

IV) Brain–computer interfaces

”The ultimate potential of machine intelligence is vastly greater than that of organic intelligence. For example even today’s transistors operate on a timescale ten million times shorter than that of biological neurons.”

A reason to doubt that superintelligence will be achieved through cyborgization, namely that enhancement is likely to be far more difficult than therapy.

Even if there were an easy way of pumping more information into our brains, the extra data inflow would do little to increase the rate at which we think and learn unless all the neural machinery necessary for making sense of the data were similarly upgraded.

V) Networks and organizations

Another conceivable path to superintelligence is through the gradual enhancement of networks and organizations that link individual human minds with one another and with various artifacts and bots.

Collective superintelligence could be one form of superintelligence. And of course Internet is the best way to ”continuing development of an intelligent web, with better support for deliberation, de-biasing, and judgment aggregation, might make large contributions to increasing the collective intelligence of humanity as a whole or of particular groups.”

”The internet stands out as a particularly dynamic frontier for innovation and experimentation. Most of its potential may still remain unexploited.”

Four types of superintelligence

There could be four types of superintelligence

A) Oracles.

B) Genies.

C) Sovereigns.

D) Tools.

A) ”An oracle is a question-answering system. It might accept questions in a natural language and present its answers as text. An oracle that accepts only yes/no questions could output its best guess with a single bit, or perhaps with a few extra bits to represent its degree of confidence. An oracle that accepts open-ended questions would need some metric with which to rank possible truthful answers in terms of their informativeness or appropriateness. To make a general superintelligence function as an oracle, we could apply both motivation selection and capability control.”

For example a pocket calculator can be viewed as a very narrow oracle for basic arithmetical questions. An internet search engine can be viewed as a very partial realization of an oracle with a domain that encompasses a significant part of general human declarative knowledge. Nick Bostrom is preferring that the first superintelligence be an oracle.

B-C) Genies and sovereigns. ”A genie is a command-executing system: it receives a high-level command, carries it out, then pauses to await the next command. A sovereign is a system that has an open-ended mandate to operate in the world in pursuit of broad and possibly very long-range objectives.

With a genie, one already sacrifices the most attractive property of an oracle: the opportunity to use boxing methods.

If one were creating a genie, it would be desirable to build it so that it would obey the intention behind the command rather than its literal meaning,

A genie endowed with such a super-butler nature, however, would not be far from qualifying for membership in the caste of sovereigns. Consider, for comparison, the idea of building a sovereign with the final goal of obeying the spirit of the commands we would have given had we built a genie rather than a sovereign.

One might think that a big advantage of a genie over a sovereign is that if something goes wrong, we could issue the genie with a new command to stop or to reverse the effects of the previous actions, whereas a sovereign would just push on regardless of our protests.

One option would be to try to build a genie such that it would automatically present the user with a prediction about salient aspects of the likely outcomes of a proposed command, asking for confirmation before proceeding. Genie-with-a-preview?”

D) ”One suggestion that has been made is that we build the superintelligence to be like a tool rather than an agent. Might one not create “tool-AI” that is like such software—like a flight control system, say, or a virtual assistant—only more flexible and capable?”

Further research would be needed to determine which type of system would be safest. The answer might depend on the conditions under which the AI would be deployed.

Three forms of superintelligence

There would be a possibility to develop:

A) Speed superintelligence.

B) Collective superintelligence.

C) Quality superintelligence.

A) ”The speed superintelligence is an intellect that is just like a human mind but faster. This is conceptually the easiest form of superintelligence to analyze. Speed superintelligence is a system that can do all that a human intellect can do, but much faster. The simplest example of speed superintelligence would be a whole brain emulation running on fast hardware. The speed of ten thousand times that of a biological brain would be able to read a book in a few seconds and write a PhD thesis in an afternoon.”

B) ”Collective superintelligence is a system composed of a large number of smaller intellects such that the system’s overall performance across many very general domains vastly outstrips that of any current cognitive system. Collective intelligence excels at solving problems that can be readily broken into parts such that solutions to sub-problems can be pursued in parallel and verified independently.”

C) ”Quality superintelligence is a system that is at least as fast as a human mind and vastly qualitatively smarter. Top-of-the-line supercomputers are attaining levels of performance that are within the range of plausible estimates of the brain’s processing power.”

How should we change according to the book?

Bostrom is predicting that there will be no more AI winter, because many institutions are heavily investing into AI. Maybe AI has already reached a point-of-no-return. Then again there might be a superintelligence winter, because it will take at least 30 years to develop human like machine intelligence. And another 50 years to develop superintelligence that ”greatly exceeds the cognitive performance of humans in virtually all domains of interest.”

The common good principle is that ”superintelligence should be developed only for the benefit of all of humanity and in the service of widely shared ethical ideals.” Maybe we should stick with that?

What should I personally do? 

Keep reading, studying and investing resources into RPA, ML and AI.

Summary

The book in six words – ”An ultraintelligent machine could design even better machines.” (I. J. Good)