Recursive Self-improvement

ultron by Alex Dempsey

Recursive [ri-kur-siv] self-improvement is the speculative ability of a strong artificial intelligence computer program to program its own software continuously. This is sometimes also referred to as ‘Seed AI’ because if an AI were created with engineering capabilities that matched those of its human creators, it would have the potential to autonomously improve the design of its constituent software and hardware. Having undergone these improvements, it would then be better able to find ways of optimizing its structure and improving its abilities further. It is speculated that over many iterations, such an AI would far surpass human cognitive abilities.

This notion of an ‘intelligence explosion’ was first described by British cryptographer I.J. Good in 1965, who speculated on the effects of superhuman machines: ‘Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an ‘intelligence explosion,’ and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make.’

A limited example that exists today are software compilers, programs that translate source code and translates it into a more basic computer language. Program language compilers are often used to compile themselves. As compilers become more optimized, they can re-compile themselves and so be faster at compiling. However, they cannot then produce faster code and so this can only provide a very limited one step self-improvement. Existing optimizers can transform code into a functionally equivalent, more efficient form, but cannot identify the intent of an algorithm and rewrite it for more effective results. The optimized version of a given compiler may compile faster, but it cannot compile better. That is, an optimized version of a compiler will never spot new optimization tricks that earlier versions failed to see or innovate new ways of improving its own program. Seed AI, however, must be able to understand the purpose behind the various elements of its design, and design entirely new modules that will make it genuinely more intelligent and more effective in fulfilling its purpose.

A ‘hard takeoff’ refers to the scenario in which a single AI project rapidly self-improves, on a timescale of a few years or even days. A ‘soft takeoff’ refers to a longer-term process of integrating gradual AI improvements into society more broadly. Computer scientists Eliezer Yudkowsky and Robin Hanson have extensively debated these positions, with Yudkowsky arguing for the realistic possibility of hard takeoff, while Hanson believes its probability is very small. Technologist Ramez Naam argues against a hard takeoff by pointing out that we already see recursive self-improvement by superintelligences, such as corporations. For instance, Intel has ‘the collective brainpower of tens of thousands of humans and probably millions of CPU cores to.. design better CPUs!’ However, this has not led to a hard takeoff; rather, it has led to a soft takeoff in the form of Moore’s law. Naam further points out that the computational complexity of higher intelligence may be much greater than linear, such that ‘creating a mind of intelligence 2 is probably more than twice as hard as creating a mind of intelligence 1.’ Programmer William Hertling replies that while he agrees there won’t be a hard takeoff, he expects that Moore’s law and the ability to copy computers may still thoroughly change the world sooner than most people are expecting. He suggests that when we postpone the predicted arrival date of these changes, ‘we’re less likely as a society to examine both AI progress and take steps to reduce the risks of AGI.’

Nanotechnologist J. Storrs Hall has said that ‘many of the more commonly seen scenarios for overnight hard takeoff are circular – they seem to assume hyperhuman capabilities at the starting point of the self-improvement process’ in order for an AI to be able to make the dramatic, domain-general improvements required for takeoff. Hall suggests that rather than recursively self-improving its hardware, software, and infrastructure all on its own, a fledgling AI would be better off specializing in one area where it was most effective and then buying the remaining components on the marketplace, because the quality of products on the marketplace continually improves, and the AI would have a hard time keeping up with the cutting-edge technology used by the rest of the world. Brazilian AI software developer Ben Goertzel agrees with Hall’s suggestion that a new human-level AI would do well to use its intelligence to accumulate wealth. The AI’s talents might inspire companies and governments to disperse its software throughout society. The AI might buy out a country and use that as its base to build power and improve its algorithms. Goertzel is skeptical of a very hard, 5-minute takeoff but thinks a takeoff from human to superhuman level on the order of five years is reasonable. He calls this a ‘semihard takeoff’.’

In a 1993 article, astrophysicist and science fiction author Vernor Vinge discussed the concept of a ‘singularity,’ i.e., a hard takeoff: ‘When greater-than-human intelligence drives progress, that progress will be much more rapid. In fact, there seems no reason why progress itself would not involve the creation of still more intelligent entities — on a still-shorter time scale.’ Vinge notes that humans can ‘solve many problems thousands of times faster than natural selection’ because we can perform quick simulations of the world in our heads. Robin Hanson collected 13 replies to Vinge, some agreeing with his singularity notion and others disputing it. In one replies, futurist Max More argued that if there were only a few superfast human-level AIs, they wouldn’t radically change the world, because they would still depend on other people to get things done and would still have human cognitive constraints. Even if all superfast AIs worked on intelligence augmentation, it’s not clear why they would better in a discontinuous way than existing human cognitive scientists at producing super-human intelligence, although the rate of progress would increase. More also argues that a superintelligence would not transform the world overnight, because a superintelligence would need to engage with existing, slow human systems to accomplish physical impacts on the world: ‘The need for collaboration, for organization, and for putting ideas into physical changes will ensure that all the old rules are not thrown out overnight or even within years.’

Creating seed AI is the goal of several organizations. The Machine Intelligence Research Institute is the most prominent of those explicitly working to create seed AI and ensure its safety. Others include the Artificial General Intelligence Research Institute, creator of the Novamente AI engine, Adaptive Artificial Intelligence Incorporated,, and Consolidated Robotics.

One Comment to “Recursive Self-improvement”

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.