Diplomacy is a seven-player strategy game in which individual nations must maneuver their armies and their relationships in order to control a majority of the game board. All pieces move simultaneously in discrete rounds, and only one piece can occupy a given province at any time.
One of Diplomacy's core mechanics is the ability to support the movements of pieces from adjacent provinces, fostering systems of collaboration towards shared goals. However, the nature of the board means players are on each others' borders very early, forcing tense interactions as nations conspire to take resources from their neighbors.
A number of articles have been written on Diplomacy over the years. In the hobby's early years, moves were relayed by mail, and were eventually accompanied by zines about social strategy, tactics, and local tournaments. The articles in these zines eventually made their way onto the web, with podcasts and YouTube channels quickly following suit.
Applying game theory to board games is nothing new – in fact, it's practically the basis of the entire field. Diplomacy writers have long mused about rational choice in the game, but it was David Rosen's 1997 paper on the subject that began to formalize a game theoretical basis for discussing Diplomacy.
With this framework, Rosen describes Diplomacy as an n-player, zero-sum game with a massive combinatorial action space and a game tree size of 10900. If that doesn't track to you, suffice to say this: Diplomacy gets out of hand a lot more quickly than chess does.
Because of this complexity, Rosen reduces the framework to a series of "sub-games." At any point, a player could be dealing with a Prisoner's Dilemma, a stag hunt, and a coin flip – it sounds like the setup to one of Danny Ocean's infamous heists. To Rosen, players focus on juggling mixed strategies to focus in on an ideal outcome and outwit their opponents.
Rosen also pays particular attention to payoffs. In a game theory class, rewards are discretely numbered and easy to calculate, and all of Diplomacy's supply centers are created equal – kind of. Portugal, for example, is much easier to hold onto than Munich, which sits in the center of the board. Because of this imbalance, different players have different priorities during any round of play, and therefore may calculate more subjective payoffs than their opponents.
While the rules explicitly describe Diplomacy as a zero-sum game, some players (known in the community as carebears) prefer to end the game by drawing with their allies instead of taking them out at the prime opportunity, as their cutthroat counterparts tend to do.
This is not the only paper applying game theory to Diplomacy – many more seek a model that falls between oversimplified payoff tables and computationally heavy, trillion-move analyses. Rosen attempts to reconcile strategic micromovements with the game at large, which he calls a "set of endless puzzles," but admits that the game is simply too complex for a basic game theoretical solution.
If Nash equilibria of a game like Diplomacy are impossible to predict and the game can't be solved, how can we get a better understanding of objective strategies? Diplomacy, by its nature, is asymmetric – certain nations are viewed as inherently stronger or weaker than others. Austria-Hungary, for example, starts the game caught between Germany, Italy, Russia, and Turkey, and is threatened with elimination from the outset. In this case, what is the best move for the empire – and, of greater importance to the mathematically minded, is it provably the best?
If Rosen's game theory cannot save us, maybe someone else's can. A group of researchers from Google's DeepMind project wrote a paper on generating a model of play for Gunboat Diplomacy, a no-press variant in which players are not allowed to communicate with one another.
This paper uses policy iteration over sampled best responses to deal with the large combinatorial action space of the game. This means that, turn after turn, the model chooses from a variety of different policies to determine ideal movesets. These policies change based on success levels of previous policies – the researchers learned that iterative improvements based on recency, rather than an average, were the best-performing models in terms of win equity.
Generating such models is terrific for narrowing down strategically ideal play. Since perfect play is impossible, and Diplomacy is easily dismissed as unsolvable, models like DeepMind are powerful tools for using deep reinforcement learning to further understand ideal, if not flawless, gameplay.
However, despite gunboat's niche popularity, Diplomacy isn't just about pure strategy and policy iteration. The best players will say that the game isn't about betrayal and power struggles – it's about relationships, cooperation and trust. And as much as Rosen might want to discount the carebears, some people still base their gameplay on external factors – "I don't want the person who betrayed me to win, and I'll do anything I can to prevent that from happening, even if it means handing the game to someone else." How can game theory, which deals with rational actors, account for the remarkable irrationality of the human mind?
Where the framework of game theory begins to deteriorate, new technologies prove that we have not yet hit the limit of idealized Diplomacy play. CICERO is Meta's AI solution to the problems posed by Diplomacy, a strategic model capable of social communication.
Communication! Of course. Strictly speaking, Diplomacy is a game of perfect information – the board is visible to all, and every player knows one another's potential moves. But secret alliances and plans abound, and CICERO is adept at sniffing out rats and exposing cracks in alliances.
Meta proudly touts CICERO as an agent that can "achieve human-level performance" in Diplomacy. This isn't strictly accurate. CICERO doesn't perform on the same level as humans – it seems to be better. It doesn't hold grudges and it makes tactical suggestions to allies with a sophisticated dialogue model and strategic planning engine.
“When playing 40 games against human players, CICERO achieved more than double the average score of the human players and ranked in the top 10% of participants who played more than one game.”
If you're unfamiliar with Diplomacy, I can't recommend Meta's research with CICERO enough. The AI agent plays the game just like many high-level players, prioritizing honesty and a methodical approach to troop movement. Tools like CICERO can help players improve their own games, and move the community as a whole closer to idealized, if not perfect, play.
Some may see the introduction of language models into a social game as unfair. Mafia and Secret Hitler just aren’t the same when you play against bots. But Diplomacy was made to be asymmetric and unfair. Russia starts out with an extra fleet, and France can easily conquer the Iberian peninsula in only a few turns. This imbalance is part of the game's charm, and leads to terrific moments when a juggernaut is felled by a ragtag team of temporary allies scrounging for resources.
Is that a game that needs to be solved? Do any of them? The pursuit of perfection can ruin the enjoyment of the hobby. The painstakingly roleplayed missives, the anxious excitement of coming treachery, and the possibility of victory through blood, sweat, and tears – will we just let the algorithms have all the fun?