How to Run the Best Board Game Tournament

Intro

Tournaments for two player games like chess are generally a solved problem. A combination of the swiss format and the chess ELO system lends itself to tournaments that can very reliably rank players based on true underlying skill.

When it comes to four player games however, this is much more of an open problem. What is the most effective way to run a tournament for a 4 player free for all game?

What makes one tournament better than another tournament?

Tournaments attempt to rank players in order of skill. The ideal outcome for a tournament is the best player finishes 1st, the second best finishes 2nd, and so forth.

In general, we care more about the top few places having an accurate ranking than any other places. All things equal, we’d give up some certainty on who the worst player is if it meant that we could increase certainty on who the best player is.

Using these insights, we can compute a tournament efficacy score or ‘loss function’ that can help us empirically evaluate different tournament structures.

Loss Function Principles

• Penalize a tournament when a worse player finishes above a better player
• If two players have equal skill level, swapping their placements in the tournament’s final ranking should have no effect on the loss function
• Because we care the most about the getting the top 4 placements correct, we add an additional penalty to tournaments that misorder these top 4 places

Constraints

The fundamental constraint when running a tournament is time. Tournaments with more games played generally are more effective than tournaments with fewer games simply by having more data points. Games can be played in parallel across rounds which helps speed things up but ultimately one player can only play one game at a time.

To compare tournaments fairly, we add the constraint that the maximum number of total tournament rounds (e.g. the maximum number of games someone can play) is 4. We’ll cover more specifics of the size and structure of our tournament in the following sections.

Methodology

To determine if one tournament structure is better than another tournament structure, we can run both tournaments with the same participants and examine the outcomes. We can compute the loss score for each tournament based on results and the tournament structure with the lower loss value is the better tournament.

Running each tournament only once gives us a terrible sample size; ideally we’d want to run each tournament hundreds if not thousands of times to give us more conviction in our results.

We can’t expect to actually run a tournament thousands of times, so that’s where we can simulate it instead!

Simulating a Tournament

First we need to define our simulated players. We can give them each a name and a skill level. The skill level can be any arbitrary number, with higher numbers corresponding to more skill.

We can base our skill number based off the chess ELO system and define a value of ~1500 as around average for our player base.

Next we need a way to simulate matches between players. Given 4 arbitrary players, we want to simulate the probability that each player finishes 1st, 2nd, and so forth.

In chess, a player that is rated 400 points higher than another player has a ~90% chance of beating them. The chess ELO system is only intended for 2 player games however and doesn’t give an equation to calculate the win rate probability in 4 player games.

We can get around this with a little bit of math and stats. Imagine we define a normal distribution with mean equivalent to a player’s skill level and variance equal to some arbitrary constant. Imagine sampling a random value from this normal distribution. This represents a single instance of this player’s performance.

This normal distribution represents the distribution of this player’s performance, with the player sometimes performing better than expected, sometimes worse, but on average they perform according to their skill level.

Now let’s introduce another player. They too would have a normal distribution corresponding to their performance, with the mean of their distribution being their true skill level.

To simulate a game between these two players, we can sample a performance number from each of the players’ normal distributions. The player who has the higher performance number is declared the winner. The person with the higher average skill level will win most of the time, but not always.

In the example above, we’d expect the orange player to win more often (~85% of the time in fact). From the random samples however, blue got lucky and happened to sample a performance value higher than the orange player, giving them the win.

This naturally extends into the 4 player case where we can sample a performance value for all 4 players and the game outcome is the rank order of the performance values. If we want to turn this into a win rate we can sample thousands of games and average together the placement data.

Now we have all the ingredients needed to simulate a tournament. We can define our set of players and simulate matches between them. For a given tournament, we can average the loss score over tens of thousands of matches and concretely say if one tournament is better than another.

Simulated Players

The ELOs of our simulated players are based off of a real tournament conducted and range from 1650 to 2355 and roughly follow a normal distribution. We have 28 players in total.

A Totally Random Tournament

For all our potential tournament structures, it’d be helpful to have a baseline to compare against. Let’s imagine the worst possible tournament. In this tournament, nobody plays any games and the final ordering of players is completely randomized.

In this totally random tournament, each player has an equal chance of finishing in any specific place. We can visualize this distribution for player one.

If this were simulated with more games, the height of the bars would all converge to the same value.

The loss score for our terrible tournament is 207.39. The magnitude of the number doesn’t matter, all that matters is that a tournament with a lower score is a better tournament.

Winner Takes All

Let’s design an actual tournament now. The principle of this tournament is everyone plays three games and the four people with the best average performance play one final game in a winner take all format (whoever wins the last game, wins the entire tournament).

After each game, we can give players 3 points for first, 2 for second, 1 for third, and 0 for fourth. To determine which players proceed to the final table, we can look at who has the most points. (How these point values were chosen will be explored later)

We also want to make it so the strongest players don’t face each other until the finals. We can do this using initial seeding. Before the first game, we can get a self professed skill level from each player. Because we have 28 players, we’ll have 28 / 4 = 7 concurrent games so the best 7 players each start off at different tables. The next best 7 players are then assigned to fill the next slot at each of those tables, and so forth.

After the first round matches are played we rank all players by the amount of points they have and assign the 7 players with the most points each to their own table. We repeat as before for the remainder of players and for the remaining rounds.

We can visualize the performance distribution for all players at the same time using a 2D heat-map.

This tournament performs significantly better than our totally random tournament and has a loss value of 31.81. One interesting feature of this tournament is the best player actually has a higher chance of finishing 5th than they do finishing 4th.

This is because if the best player makes the final table, the chance that they finish last is extremely low. It’s a higher chance that they don’t make the final table at all. This effect is magnified by the fact that there are 28 total players but only 4 can make the final table. If this were a 16 person tournament, we’d expect this distribution to be more monotonic.

Random Assignment

Winner takes all in the final game sounds nice in theory but it’s actually a bad idea from a statistical standpoint. It means that when ranking the top 4 players, there is only one data point to consider. It makes more sense to have the initial three games of the top 4 players also factored in.

Let’s consider a new tournament structure where each player plays 4 games against totally random opponents. At the end of the 4 games, the winner is the player with the most points.

This gives us a loss of 28.85, outperforming the previous tournament by not using the ‘winner takes all’ format.

Swiss

The previous random tournament paired players against totally random opponents each round. We can improve upon the tournament by adopting the typical swiss paradigm of pairing players against opponents of equal win/loss record.

To decide the tables for a round, take the current sorted ranking of players and pair the top 4 into one table, the next 4 into the next table, and so forth. After each round, allocate points from results and then re-sort the player ranking before computing the next set of matches.

This gives us a loss of 28.3. A marginal improvement but an improvement nonetheless. The placement distributions look very similar to the random swiss so the graphs are omitted for brevity going forward.

Optimizing the Point Distribution

How many points should be given out for first, second, third, and fourth place finishes? The best turns out to be 3 points for first, 2 for second, 1 for third, and 0 for fourth. Shorthand this can be expressed as [3,2,1,0].

There are actually multiple optimal point assignments, multiplying [3,2,1,0] by any constant or adding any constant also results in an optimal point assignment. The only property that must hold is that the difference in points between 1st and 2nd is the same as the difference between 2nd and 3rd and so forth through 4th.

Because it’s aesthetically nice to get 10 points for first, that gives us our preferred optimal point distribution of [10, 7, 4, 1]. This is the point distribution we’ve used throughout this analysis.

Strength of Schedule

Using our point distribution, if someone placed 1st then 4th over two games, their cumulative score would be 10+1 = 11 points. If someone placed 2nd then 3rd over two games, their cumulative score would be 7+4 = 11 points as well.

There are lots of opportunities for two players to have a tied score so it would be nice if we had a meaningful tiebreaker of some sort. In comes strength of schedule. This is a commonly used tiebreaker in sports and essentially says that for two teams that have equal scores, the team that has had to battle stronger opponents to get that score is likely stronger.

We can tangibly apply this by averaging the cumulative points across all opponents a player has faced up until the current point in the tournament. This number is then used as a tiebreaker if their normal score is tied.

Using strength of schedule in the above more intelligent swiss format gives us a new loss value of 27.77.

ELO Instead of Points

In chess, each player has an ELO rating which attempts to reflect each player’s true underlying skill. A player in the international chess federation starts off with a rating of 1,000 points and gains and loses points based on their performance in matches.

There’s a wonderful paper from Microsoft Research that adapts the chess ELO system to allow games with a variable number of teams. We can use this methodology to introduce an alternative to the point based ranking system in our tournaments.

Instead of assigning points for 1st, 2nd, 3rd, and so forth, we can initialize all players to an ELO rating of 1500. After each match, we perform the ELO update algorithm and the standing of players is the sorted ranking of their ELOs. In the fixed point based system of before, beating the best player in the tournament and beating the worst player in the tournament give you an equal number of points. Now, the ELO system rewards you more for defeating more skilled players.

Using this ELO rating system brings our loss down to 26.93.

Seeding Pure Swiss

The core tenant of swiss tournaments is that at every point in the tournament, players of equal win/loss record are matched up against each other.

This means that in the first round of the tournament, the top 4 seeds all play against each other. More often than not, the 4th best player is starting off the tournament with a 4th place. This is pretty devastating and less skilled players can get a much easier win at their tables and be in a much stronger position with cumulative points.

If we have all 28 players in seeded order, we can represent table assignments using the shorthand [1111222233334444555566667777] which means that the first four players are assigned to table 1, the next four players are assigned to table 2, and so forth.

In single elimination brackets, seeds are used so that the strongest opponents don’t all face each other in the first round. A similar principle can be applied here. With a 28 person tournament there are 7 games so the top 7 seeds can each be assigned to a separate table. The next 7 can be assigned to fill the second slot at each table, and so forth. This ensures the average skill level at each table is more consistent. The shorthand for this assignment is [1234567123456712345671234567]

Blended Swiss

With seeded swiss, after the first seeded round, players face off against players who have all won their first game. In blended swiss, the top table will have two players who won their first game and two players who finished second in their first game. This makes the table a little bit easier for the top two players. The shorthand for this is [1212121234343434565656567777]. Using pure seeding assignment for round 1, this blending approach for round 2, and pure swiss for rounds 3 and 4 gives us a new best in class loss of 26.19. This is the best result we were able to achieve.

Interestingly, seeding the first round of swiss when using the ELO system improved the loss score but when using the point based system for 1st, 2nd, 3rd, and 4th, unseeded swiss performed better.

Round Robin

One interesting observation throughout this research is that if you have a 5 round tournament with 16 people, there is a configuration such that everyone can play everyone exactly once over the 5 rounds.

It was hypothesized that this would be the best tournament structure because it maximized the amount of unique matchup pairings. It turns out however that this tournament performs quite poorly and that swiss is still a better system.

Attempting to prevent players from playing against the same opponents multiple times makes the tournament worse from a statistical standpoint.

Seeding

Some tournament structures are very sensitive to initial seeding. It’s not realistic to be able to seed a tournament perfectly, but asking players how good they think they are gives a pretty strong signal and leads to surprisingly accurate seeds.

In each of our simulations we randomly varied the seeding by adding a random offset to the player’s true skill. This random offset was sampled from a normal distribution and the skills were then sorted with the presence of that random offset.

This resulted in seeding that would roughly mirror player’s self professed skill levels.

3 vs 4 Player Games

With a 4 player board game tournament, the majority of the time the total number of players will not be evenly divisible by 4. For this reason, some games might only have 3 players. Most of the logic stays the same, with a few changes. When assigning points for 1st, 2nd, 3rd, and so forth with four players, we use [10, 7, 4, 1], with three players we should use [10, 6, 1]. The games with 3 players should also be allocated for the least skilled players (lowest initial seeds).

Real Life Tradeoffs

The tournament that is most statistically optimal ends up being pretty difficult to actually run in practice without existing software. It’s nontrivial to perform the 4 player ELO updates and it’s a confusing system for the players involved if they don’t understand the math.

Additionally, computing strength of schedule for tiebreakers can be troublesome.

For this reason, we’d advise just using the [10, 7, 4, 1] point system when running tournaments in practice, breaking ties by initial seeding. It’s good enough for the sake of logistical simplicity.

Tournaments from an Information Theory Perspective

One last interesting thought is that tournaments have a lot in common with sorting algorithms. Tournaments are effectively just fuzzy sorting algorithms that are able to be parallelized in a unique way.

From this insight, we can apply some established information theory principles to our tournament math. A regular list takes nlogn comparisons to sort the list completely. This means for a 28 person tournament, we need at minimum 134 comparisons to completely sort the list.

Each game is effectively 6 pairwise comparisons between the players involved. With 28 people, 7 games can be played in parallel so each round gives us a total of 7 6 = 42 comparisons. So at minimum, a 28 person tournament needs 4 rounds (42 comparisons 4 comparisons per round = 164 comparisons) to achieve the minimum necessary 134 comparisons.

This information theory estimate doesn’t take into account fuzzy sorting so more comparisons are needed to increase accuracy further when the comparison between two items isn’t certain.

Summary

The most effective tournament structure for a 4 round, 28 player splendor tournament is blended swiss using ELO for reseeding. This tournament structure gives a 99.76% chance that the best player finishes top 4 and an 85.3% chance they win the whole tournament.

ELO calculation is hard to perform on the fly so the best performing tournament without ELO is an unseeded true swiss using strength of schedule as a point tiebreaker. This gives the best player a 99.46% chance of finishing top 4 and an 84.6% chance of winning the whole thing.

Larger Tournaments

All above analysis was performed on a 28 person tournament. Below is a summary of the result table using a 64 person tournament assuming a uniform distribution of underlying skills.

Happy gaming!