Deepnash
Developing intelligent agents that learn end-to-end to make optimal decisions under imperfect information in Stratego, from scratch, with no demonstration data, is therefore the challenge Deepmind AI researchers set themselves. To do so, they chose the most complex version of the game: Stratego Classic and introduced Deepnash, an autonomous agent that beat the best existing methods and reached the level of an expert game.
DeepNash relies on a principled, model-free RL algorithm called Regularized Nash Dynamics (R-NaD) combined with a deep neural network architecture to converge to an epsilon-Nash equilibrium (The notion of Nash Dynamics refers to mathematician John Forbes Nash in game theory).
A Nash equilibrium guarantees that the agent will work well, regardless of the opponent. It is often used to play against humans in two-player zero-sum games.
The Regularized Nash Dynamics (R-NaD) algorithm, based on the concept of regularization, implemented via the deep neural network, thus converges to an approximate Nash equilibrium, instead of "cycling" around it, and directly modifies the underlying multi-agent learning dynamics. (figure 1b)
Evaluation of Deepnash
The team then evaluated Deepnash's performance on the Gravon platform, an online game server well known to Stratego players where it was measured against eight AI bots. The table below shows its efficiency as it won 97% of the challenges:
It was also tested for two weeks last April against the best human players and won 84% of the 50 games, placing it in 3rd place in the Stratego 2022 Classic challenge. Deepnash has thus demonstrated its capabilities for deployment, bluffing and compromise.
DeepNash could unlock further RL applications for real-world multi-agent problems characterized by imperfect information that are currently beyond the reach of state-of-the-art AI methods.
Article source:
ArXiv: "Mastering the Game of Stratego with Model-Free Multiagent Reinforcement Learning" (arXiv:2206.15378)
Julien Perolat , Bart de Vylder , Daniel Hennes , Eugene Tarassov , Florian Strub , Vincent de Boer , Paul Muller , Jerome T. Connor , Neil Burch , Thomas Anthony , Stephen McAleer , Romuald Elie , Sarah H. Cen , Zhe Wang , Audrunas Gruslys , Aleksandra Malysheva , Mina Khan , Sherjil Ozair , Finbarr Timbers , Toby Pohlen , Tom Eccles , Mark Rowland , Marc Lanctot , Jean-Baptiste Lespiau , Bilal Piot , Shayegan Omidshafiei , Edward Lockhart, Laurent Sifre , Nathalie Beauguerlange , Rémi Munos , David Silver , Satinder Singh , Demis Hassabis , Karl Tuyls.
Translated from Deepmind AI présente « DeepNash », l'agent autonome RL sans modèle, expert du jeu « Classic Stratego »
Our articles will then appear first in Google Top Stories.
