At the same time that the United Nations Security Council was debating global AI risk governance, I was on the other side of Manhattan at a protest outside of Google’s NYC office. High-level policy conversations can help, but prior experience on nuclear war and climate change shows that grassroots citizen activism is also essential to advancing responsible risk policy.
In a companion article, I analyze the current risks posed by AI, covering nuclear war, cybersecurity, and takeover catastrophe. Here, I discuss current AI risk governance challenges and what to do about them, with a focus on the US.
The problem is twofold. First, the US frontier AI companies such as OpenAI and Anthropic are acting in a deeply irresponsible manner. They have systematically failed to safely manage their own products, resulting in numerous illicit cyberattacks. They also promise to develop even more dangerous products in the immediate future. And they have an institutional culture that is fundamentally antithetical to responsible risk management.
Second, the Trump administration and some of its Congressional allies have rejected any AI risk governance. Instead, they are calling for corporate self-regulation, which is clearly not working, continued development to outpace China, which is reckless, and aggressive government adoption of AI, which is elevating the risk of war, including nuclear war.
The path forward is clear:
• The US frontier AI companies should immediately stop all development and testing of new AI systems, to be resumed only after rigorous, externally verifiable safeguards are in place. They should do this on their own as a minimal step toward avoiding serious harms and earning basic trust.
• Employees at these companies should focus on pushing for this, and if they can’t, they should quit and push from the outside.
• Investors should likewise insist on it or sell their stake in the companies.
• The US federal government should impose regulations and liability with strict enforcement to ensure universal compliance among American AI companies. If need be, shut the companies down. In addition, the US government should reach out to other countries, especially China, to do the same for their frontier AI companies. The US government should do this as part of a broader initiative to address the harms created by frontier AI companies. Finally, the US government should manage its own use of AI more responsibly, especially in the military.
• US state governments should do what they can within the scope of their authority.
• US citizens from across the political spectrum should stand up and demand action from both the companies and the government. This may be the crucial activity, especially to change the Trump administration’s position.
• Media outlets should ensure that citizens have the information needed to get involved.
• Citizens and governments outside the US should contribute as they are able to, such as by amending their laws to create legal penalties for cyberattacks caused by acts of corporate negligence.
As my colleagues and I have previously documented, there are opportunities for a wide range of people to contribute to AI corporate governance.
The case for stopping the development and testing of frontier AI systems is robust. It is an appropriate response to the cybersecurity incidents that have already occurred. The incidents involve breaches that, had they been done intentionally by humans, would constitute crimes. Companies should not develop products that are known to function in quasi-criminal ways and governments should not allow them to do so.
Stopping development and testing also addresses risks from more advanced future AI systems. These risks are deeply uncertain and fiercely contested within expert communities. My own analysis leaves me skeptical of at least some of these scenarios, but there is a lot of uncertainty and I cannot rule them out. Stopping development and testing addresses these scenarios regardless of how probable they may be.
Finally, stopping development and testing creates space to establish responsible risk governance. The AI companies’ “Wild West” culture of rushing to deliver new products must end. The companies’ practice of not engaging with subject matter experts on the risks their products may create must end. The companies’ insistence on continuing to build products that they themselves believe—rightly or wrongly—may cause extreme harm must end. Whatever the actual risks from the companies’ products may be, it is clear that the companies themselves have a deeply irresponsible set of practices. Stopping development and testing ensures that the companies do not cause major harms while responsible risk governance is put in place.
OpenAI recently announced that it is stopping the development of new AI systems. Good. But that is its second stoppage in just three months. This on-again-off-again approach isn’t enough.
Remarkably, the US frontier AI companies are calling for government intervention, including at the Security Council. This does not in any way let them off the hook for their central role in causing the problem in the first place, or the myriad other harms from their work. We should also be vigilant of the potential for them to use government regulation to shield themselves from competition. Nonetheless, calling for government intervention is, in my opinion, commendable, a stark contrast with (for example) fossil fuel companies lobbying against climate policy.
To the extent that the companies’ calls for government intervention are honest, they are speaking to the need for collective action, the idea that individual companies cannot solve the entire problem on their own and are not incentivized to do so. This is a legitimate concern. As a prior GCRI research study explains, there are three major types of solutions for achieving collective action: government regulation, private ownership, and community self-organizing. For the case of AI competition, private ownership is less helpful because there is no one owner that can compel collective action. Community self-organizing has potential, but AI companies are failing at it and there may be limits to how much they can do under US anti-trust law. The companies should do more, acting unilaterally if need be, but government regulation has a central role.
The US federal government needs to take the lead. Many of the AI companies that are creating the risks are based in the US. Individual US states have advanced some helpful AI risk policies, such as California’s SB 53 and the New York RAISE Act. Most recently, the Attorney General of Florida has requested a temporary injunction against OpenAI. However, the federal government has more operational capacity to regulate AI companies and more legal authority for both domestic and international governance activities, especially via the Commerce Clause and the Limits of the States clauses of the US Constitution.
Unfortunately, the White House is on the wrong side of the issue. Despite bipartisan calls for action, President Trump has openly rejected AI risk policy, even calling the risk a “hoax”. This is consistent with the administration’s longstanding efforts to block AI risk governance. In June, the administration briefly reversed course, imposing restrictions on some frontier AI models due to cybersecurity concerns. At the time, I described it as a potential turning point for AI risk governance and an encouraging development. However, the administration has since relapsed back to being part of the problem.
The White House is also ultimately responsible for the US government’s use of AI, especially in the military. Overly aggressive military adoption of AI, combined with a more general hawkish stance, contributed to the tragic February 28 attack on an Iranian school and a near-miss China-US war. This posture increases the risk of more tragic accidents, including the risk of nuclear war. This posture is attributable to Secretary Hegseth, presumably with White House support.
This is where grassroots citizen activism comes in. Congress and the White House must be pushed to act. Citizen pressure has already worked on AI data centers, albeit so far only for local policy decisions. For federal policy, the best benchmark may be the 1980s nuclear peace movement, which pushed the Reagan administration to reverse its hawkish, anti-Soviet stance on nuclear weapons and instead embrace diplomacy and arms control. The situation with AI today is remarkably similar, replacing Reagan with Trump and the Soviet Union with China.
One important difference between the 1980s nuclear peace movement and the AI situation today is that, in the 1980s, Americans were much more adept at civic participation. Following a multi-decade decline, Americans today are tragically out of practice. A primary reason for this is that policy advocacy communities have shifted from building mass citizen coalitions to pursuing insider influence. AI policy advocacy communities—including GCRI—fit within this trend; now we lack grassroots political muscle at a time when that may be the one thing that could move the needle.
There are a few dedicated AI civic advocacy organizations, but they all have limits. Pause AI, Stop AI, and Stop the AI Race seem designed to appeal to the narrow group of people focused on AI safety, whereas mass appeal is needed to change policy. Two groups hold more potential for mass appeal: Humans First, founded by former Tea Party leaders and catering to the political right, and Irreplaceable, founded by former climate organization leaders and catering to the political left. At first, both initiatives operated as Humans First, but partisan divisions prompted the left flank to split off. Now there is a need for an organization for people who don’t identify with either political side. Humans First also uses an “American First” framing that conflicts with the need for international diplomacy, including with China. Neither organization permits concerned citizens to start local chapters. (To its credit, Pause AI does.) Research on political organizing shows that decentralized structures tend to outperform top-down structures at engaging the public and changing policy.
Growing new civic advocacy organizations takes time. Meanwhile, organizations built for other issues can contribute. Climate and environment groups have been especially active, for example constituting a large portion of the Stop Data Centers Coalition. Indeed, it was through my connections to local climate groups that I learned of the Google protest shown above. Engaging with organizations built for other issues is additionally appropriate because AI is not the only important civic issue. We as a society must be able to address AI alongside other issues.
The clock is ticking. The longer we let frontier AI companies continue to develop and test new AI systems, the larger a risk we incur. Ditto for the US government using AI irresponsibly. The recent success of data center activism shows that the American people are still able to move policy, and to do it fairly quickly. We need to adapt this for other AI risks to keep the world safe from reckless frontier AI companies and the current US government.
