The creation of this open letter was led by Tolga Bilge, working as part of a team with great amounts of help from Eli Lifland, Jonathan Mann, Molly Hickman, Siméon Campos, and Akash Wasil. Our web developer is Philip Hall Andersen.

We are motivated by the importance and urgency of avoiding an AI catastrophe, and hope this letter will advance the conversation on how to reduce this risk. You can contact us at 

About

We call on governments worldwide to actively respond to the potentially catastrophic risks posed by advanced artificial intelligence (AI) systems to humanity, encompassing threats from misuse, systemic risks, and loss of control. We advocate for the development and ratification of an international AI treaty to reduce these risks, and ensure the benefits of AI for all.

Leading experts, including Geoffrey Hinton, Yoshua Bengio, and the CEOs of OpenAI and Google DeepMind, have 

 about the catastrophic risks posed by AI, and called for the reduction of this risk to be treated as a global priority. Similarly, AI experts have called for 

a pause on the advancement of AI capabilities

. In recent months, world leaders have increasingly drawn attention to the need to ensure that AI is developed safely.

 that AI could lead to human extinction or a similarly catastrophic curtailment of humanity's potential. We consider that the gravity of these risks warrants immediate and serious attention.

We believe the central aim of an international AI treaty should be to prevent the unchecked escalation of the capabilities of AI systems while preserving their benefits. For such a treaty, we suggest the following core components:

: Internationally upheld thresholds on the amount of compute used to train any given AI model, with a procedure to lower these over time to account for algorithmic improvements.

: A collaborative AI safety laboratory akin to 

 for pooling resources, expertise, and knowledge in the service of AI safety, and acting as a cooperative platform for safe AI development and safety research.

: Enable access to the APIs of safe AI models, with their capabilities held within estimated safe limits, in order to reduce incentives towards a dangerous race in AI development.

: An international commission responsible for monitoring treaty compliance.

Crucially, the effectiveness and impact of such a treaty hinges on widespread agreement across the international community. These measures, combined, promote a safer exploration of AI's immense potential by fostering a cooperative, safety-first approach to AI research and development.

Such an AI treaty has the potential to play a role similar to the International Atomic Energy Agency (IAEA), which was formed to manage nuclear risks and promote the peaceful uses of nuclear energy. Similarly, an AI treaty could not just reduce risks from AI, but also ensure that the benefits of AI are accessible to all. The potential of AI extends far beyond our current understanding and should be viewed as a common good. It is essential that this extraordinary resource benefits all of humanity.

Humanity has shown remarkable unity when faced with global threats, as demonstrated by international cooperation on nuclear non-proliferation. We believe the risks posed by AI systems warrant at least as much caution and coordination.

We urge members of the international community to actively engage in discussions around an AI treaty, and strive towards implementing a robust set of international regulations. We advocate for the formation of a working group, with broad international support, to develop a blueprint for such a treaty. This responsibility does not rest solely on a few shoulders, but on the collective strength of the global community. Our future hangs in the balance. We must act now to ensure AI is developed safely, responsibly, and for the betterment of all humanity.

“AI is going to change the way we teach, learn, and solve challenges. But in seizing this moment, we must manage the risks.”



European Commission President Ursula von der Leyen, 

"We have a narrowing window of opportunity to guide this technology responsibly."

“We need to jointly fend off risks, and develop AI governance frameworks and standards with broad-based consensus, so as to make AI technologies more secure, reliable, controllable and equitable.”

“People will be concerned by the reports that AI poses existential risks, like pandemics or nuclear wars. I want them to be reassured that the government is looking very carefully at this.”

We Can Prevent AI Disaster Like We Prevented Nuclear Catastrophe

International Institutions for Advanced AI

Matthijs M. Mass, José Jaime Villalobos, 

International AI Institutions: A Literature Review of Models, Examples, and Proposals

Artificial Intelligence Is Stuck. Here’s How to Move It Forward.

An International Treaty to Implement a Global Compute Cap for Advanced Artificial Intelligence

Treaty on Artificial Intelligence Safety and Cooperation

, a group of expert forecasters, have drafted and analyzed the efficacy of an example treaty, 

, that proposes measures to reduce the chance of an AI catastrophe. 

 finds a significant chance of a global AI catastrophe in the coming decades, and suggests a strong AI treaty akin to the TAISC could halve this risk.

Urging an International AI Treaty: An Open Letter

Welcome to aitreaty.org. We respect your privacy and are committed to protecting it. This statement governs our privacy policies with respect to those users of the site ("Visitors") who visit without transacting business and Visitors who register to sign the open letter.

On our site, we only collect personal information that you voluntarily provide to us, which includes your name and email address, for the sole purpose of adding your signature to the open letter. No other personal information is collected from visitors of our website.

We use your personal information for the following purpose:

To add your signature to the open letter.

We do not use cookies for tracking purposes.

Under the General Data Protection Regulation (GDPR), individuals have a number of important rights. These include the right to be informed about the collection and use of their personal data, the right of access, the right to rectification, the right to erasure (also known as the ‘right to be forgotten’), the right to restrict processing, the right to data portability, the right to object, and rights related to automated decision making including profiling.

We adhere to these principles and respect your rights under the GDPR if you are a resident of the European Economic Area (EEA) or the United Kingdom.

Similarly, if you are a resident of California, USA, we adhere to the California Consumer Privacy Act (CCPA). If you are a resident of Canada, we adhere to the Personal Information Protection and Electronic Documents Act (PIPEDA). If you are a resident of Virginia, USA, we adhere to the Consumer Data Protection Act (CDPA).

If you wish to be informed about what Personal Data we hold about you and if you want it to be removed from our systems, please contact us.

We do not sell, trade, or exchange your personal information with third parties.

We may update our Privacy Policy from time to time. You are advised to review this Privacy Policy periodically for any changes.

If you have any questions about this Privacy Policy, please contact us:

Privacy Policy

Professor / Scientific Director

Yoshua Bengio

Professor

Bart Selman

Max Tegmark

Gary Marcus

Yi Zeng

Senior Research Scientist

Victoria Krakovna

President

Nell Watson

Geoffrey Odlum

Co-Founder of Skype

Jaan Tallinn

Artist

Claire Boucher 

Distinguished Professor Emeritus

Huw Price

Senior Program Officer

Luke Muehlhauser

Professor Emeritus

Daniel Dennett

Joseph S. and Sofia S. Fruton Professor of Astronomy & Physics

Priyamvada Natarajan

Alexander von Humboldt Professor of AI

Holger Hoos

Analyst, AI Governance

Felicity Reddel

Senior Research Fellow

Toby Ord

Executive Director

Anthony Aguirre

CEO & Research Scientist

Andrew Critch

ML Researcher, Alignment Research Center Evaluations

Daniel Ziegler

Ramana Kumar

Director, UNSW Business AI Lab

Mary-Anne Williams

Professor of Computer Science and Director of Centre for Digital Innovation

The Anh Han

Co-founder of Writely (Google Docs), founder of Scalyr

Steve Newman

Jeffrey Ladish

Professor/Principal Advisor

Paul Nemitz

Director of the Transatlantic Program

Balkan Devlen

Connor Leahy

Tristan Harris

Senior Research Fellow, AI & Law

Matthijs Maas

Associate Professor of Philosophy, Linguistics, and Psychology

Alexis Wellwood

Privacy Policy

Table of Contents

Welcome

Personal Information

Use of Information

Cookie Policy

Data Protection Rights

Third-party Disclosure

Changes to this Privacy Policy

Contact Us