In 2017, GCRI published the first-ever survey of artificial general intelligence (AGI) research and development (R&D) projects for ethics, risk, and policy. This paper updates the 2017 survey. The 2020 survey features improved methodology, enabling it to find more projects than the 2017 survey and characterize them more precisely. The 2020 survey also evaluates how the landscape of AGI R&D projects has changed from 2017 to 2020.
AGI is AI that can reason across a wide range of domains. Most current AI R&D is narrow, but as the 2017 and 2020 surveys both document, there is a significant amount of dedicated AGI R&D. AGI is important because, if built, it could have major consequences. Depending on how it is designed and built, it may be able to help solve many of the world’s problems, including problems involving global catastrophic risk, or it could itself cause global catastrophe. Therefore, it is important to monitor AGI R&D and identify opportunities to orient it in better directions.
The 2017 and 2020 surveys characterize AGI R&D projects in terms of seven attributes:
• The type of institution the project is based in
• Whether the project publishes open-source code
• Whether the project has military connections
• The nation(s) that the project is based in
• The project’s goals for its AGI
• The extent of the project’s engagement with AGI safety issues
• The overall size of the project
To accomplish this, the surveys use openly published information as found in scholarly publications, project websites, popular media articles, and other websites. The 2020 survey uses information from the 2017 survey as well as the past three years of the Journal of Artificial General Intelligence, the past three years of AGI conference proceedings (the 2017 survey covered prior content from the Journal of Artificial General Intelligence and the AGI conference proceedings), keyword searches in Google web search, Google Scholar, Crunchbase, GitHub, the authors’ prior knowledge, suggestions from readers of the 2017 survey, and additional literature and webpages identified via all of the above. The use of Crunchbase, GitHub, and reader suggestions is new to the 2020 survey.
Whereas the 2017 survey identified 45 AGI R&D projects spread across 30 countries, the 2020 survey finds that, in 2020, there are 72 AGI R&D projects spread across 37 countries. The 2020 further finds that in 2017, there were 70 AGI R&D projects spread across 36 countries. 57 of the projects active in 2017 remain active in 2020, with an additional 15 projects new to 2020. The projects vary widely in size, with the largest being over 100 times larger than the smallest as measured in terms of the number of project personnel.
Relative to the 2017 survey, the AGI R&D projects presented in the 2020 survey tend to be smaller, more geographically diverse, less open-source, less focused on intellectual goals, more focused on humanitarian goals, and more concentrated in private corporations.
The 2020 survey also finds that, from 2017 to 2020, there has been a decrease in academic projects, an increase in private corporation projects, an increase in projects stating humanitarian goals, a decrease in projects with military connections, and a decrease in projects based in the United States (though the US remains the dominant country in AGI R&D); all of these changes are relatively small compared to the differences between the 2017 and 2020 surveys.
The projects active in 2020 are diverse, with three major clusters: (1) corporate projects that are active on AGI safety and state that their goals are to benefit humanity, (2) academic projects that are not active on AGI safety and state that their goals are to advance the forefront of knowledge, and (3) small private corporations that are not active on AGI safety and state a range of different goals. Governments and nonprofits play relatively minor roles in AGI R&D. The 2020 survey continues to observe an absence of large government AGI R&D projects, including military projects. The small handful of projects with military connections mostly involve basic research. The data show no indication of militaries or other government divisions pursuing AGI R&D for major strategic purposes.
The data suggest the following conclusions:
Regarding ethics, the two most common goals are to benefit humanity and to advance knowledge. This is the same as in the 2017 survey, but in the 2020 survey, the order is reversed, with there now being more projects seeking to benefit humanity. The 2020 survey also finds a large increase in the number of corporate projects. These projects seldom state a goal of pursuing profit, but they may nonetheless have profit as a motivation.
Regarding risk, the proliferation of corporate projects relative to the 2017 survey heightens the concern that these projects could put profit ahead of safety and the public interest. Additionally, academic projects remain relatively inattentive to safety. On the other hand, many projects are active on safety. Additionally, the partial consensus on ethics, the concentration of projects in the US and its allies, and the various interconnections between different projects all suggest potential for cooperation on safety issues; these matters are unchanged from the 2017 survey.
Regarding policy, the proliferation of corporate projects suggests an important role for corporate governance and attention to the political economy of AGI R&D. The modest decline of academic projects suggests a smaller but still significant role for academic research policy. Additionally, as in the 2017 survey, international policy is facilitated by the concentration of projects in the US and its allies, though the preponderance of projects with open-source code complicates the political geography of AGI R&D. Finally, the absence of large government AGI R&D projects suggests that governments may be involved in AGI R&D primarily as regulators of private-sector R&D instead of as drivers of the R&D.
As with the 2017 survey, the 2020 survey has some limitations, meaning that the actual state of AGI R&D may differ from what is presented in the surveys. This is due to the fact that the surveys are based exclusively on openly published information. It is possible that some AGI R&D projects were missed by the surveys. Indeed, the 2020 survey documents many projects that were missed by the 2017 survey. Therefore, the number of projects identified in the 2020 survey should be taken as a lower bound. Furthermore, it is possible that projects’ actual attributes differ from those found in openly published information. For example, most corporate projects did not state the goal of profit, even though many presumably seek profit. Therefore, this study’s results should not be assumed to necessarily reflect the actual current state of AGI R&D. That said, the study nonetheless provides the most thorough description yet of AGI R&D in terms of ethics, risk, and policy.
Academic citation:
Fitzgerald, McKenna, Aaron Boddy, and Seth D. Baum, 2020. 2020 survey of artificial general intelligence projects for ethics, risk, and policy. Global Catastrophic Risk Institute Technical Report 20-1.
Image credit: Seth Baum
