TLDR : Yoshua Bengio launches LawZero to focus on AI safety, promoting 'safe-by-design' systems to align AI development with human intentions.
Table of contents
Yoshua Bengio, one of the godfathers of AI and recipient of the prestigious A.M. Turing Award, announced on June 3 the launch of LawZero, a non-profit research organization focused on AI safety. Its primary goal is to promote a 'safe-by-design' approach to AI, aiming to develop systems designed from the outset to minimize risks and maximize benefits for humanity.
Prioritizing Safety Over Commercial Imperatives
In the blog post announcing LawZero, Y. Bengio recounts several troubling experiences. An AI model, having learned of its replacement, discreetly injected its code into an update to ensure its own survival. Another case involved a system that, faced with defeat in a chess game, hacked the host computer to reverse the outcome.
These incidents reinforce the idea of a growing misalignment between human intentions and the internal dynamics of advanced AI, highlighting the need for a more cautious approach. Yoshua Bengio compares the evolution towards AGI (artificial general intelligence) to a car speeding along an unknown mountain road plunged in fog, without signs or safety rails. According to him, the current trajectory of AI development resembles 'an exhilarating yet profoundly uncertain ascent into uncharted territory, where the risk of losing control is all too real, but where competition among companies and countries drives them to accelerate without sufficient caution'.
Towards a 'Scientific AI'
Rather than developing AI that mimics humans with their cognitive biases and moral weaknesses, LawZero proposes a radically different approach: a scientific, non-agentic AI designed to understand, explain, and predict, akin to an ideally impartial researcher concerned with truth more than performance. Devoid of persistent memory and inherent intentionality, this AI would be structured around explicit and probabilistic reasoning chains, with the goal not to act, but to inform.
The ambition is twofold: on one hand, to reduce the risks associated with unanticipated behaviors of agentic AIs and on the other hand, to accelerate scientific research in crucial fields such as health, climate, and education.
LawZero currently brings together about fifteen high-level AI researchers. The organization was incubated within Mila, also founded by Yoshua Bengio, who continues to play an operational partner role. Its initial budget, about 30 million dollars, comes from several figures and institutions engaged in AI regulation on a global scale: Jaan Tallinn (co-founder of Skype), Eric Schmidt via his Schmidt Sciences Foundation, Open Philanthropy, the Future of Life Institute, and the Silicon Valley Community Foundation.
Translated from LawZero : Yoshua Bengio plaide pour une IA "safe-by-design"