Meta's Cicero might bring some danger for politicians in the upcoming years

Ever since IBM's Deep Blue artificial intelligence system defeated world chess champion Garry Kasparov at his own game in 1997, humanity has watched, haplessly, year after year as our code-based underlings vanquish us in ever more complicated games. There's a catch though. While AI bots increasingly excel at trumping humans in head-to-head adversarial board and video game matches, the systems typically fare far worse when instructed to cooperate with other humans to accomplish a shared task. Researchers at Meta believe their new "Cicero" AI may have something to say about that.

For the first time, according to a new study shared with Gizmodo by Meta's Fundamental AI Research Diplomacy Team, researchers trained an AI to attain "human-level performance" in the war strategy board game Diplomacy. The new AI agent, named after the classical statesman and scholar who witnessed the fall of the Roman Republic, was able to effectively communicate and strategize with other human players, plan the best methods for victory, and in some cases, even pass as a human. Now, the researchers say Cicero, which accomplished its tasks by combining dialogue and strategic reasoning models, stands as a "benchmark" for multi-AI agent learning.

Over the past twenty or so years, an increasingly impressive assortment of AI systems from a variety of companies have trounced human players in everything from Chess and Go to more modern games like Starcraft. While different in content, those games all share one fundamental similarity: they are all a zero-sum, winner takes all competition.

Diplomacy is different. In Diplomacy, seven players compete to control the majority of supply centres. Players are constantly interacting with each other and each round begins with a series of pre-round negotiations. Crucially, Diplomacy players may attempt to deceive others and may also think the AI is lying. Researchers said Diplomacy is particularly challenging because it requires building trust with others, "in an environment that encourages players to not trust anyone." In other words, for an AI to "win" at Diplomacy it needs to both understand the rules of the game efficiently also fundamentally understand human interactions, deceptions, and cooperation, and know how to string together sentences without sounding like a malfunctioning dishwasher.

Cicero more or less did that. Meta says Cicero more than doubled the average score of human players across 40 anonymous online Diplomacy games and ranked in the top 10% of players who played more than one game. Cicero even placed 1st in an eight-game tournament involving 21 participants. At every stage of the game, Cicero would model how other competing players were likely to act based on their game performance and text conversations.

The researchers conducted their study experiment on August 19 and October 13 2022 over 40 anonymized online weDiplomacy.net games totalling 72 hours of play. Researchers say they didn't see in-game messages to suggest human players believed they were playing against an AI. Cicero apparently "passed as a human player," in 40 games of Diplomacy with 82 unique players. In one case highlighted in the study, Cicero was able to successfully change a human player's mind by proposing a mutually beneficial move.

Cicero was trained on a hefty chunk of previous Diplomacy data in order to get it to prepare it to properly communicate with other players. The researchers say the AI was trained on a dataset of 125,261 anonymized Diplomacy games, around 40,000 of which contained dialogue. In total, that dataset contained over 12 million messages exchanged between human players.