Artificial Intelligence Algorithms and Kubernetes IV

How using concurrency will result in better response times

In one of our previous posts, we created a planning algorithm, and used it to simulate a game of Warhammer (or at least a similar wargame). But the actions we considered for that problem were too generalist. These actions might work as a general guide for any game. But it would be more interesting if we had tailored actions for the factions we were playing, or at least some statistical data about the outcome of the actions, right?

In this post, we will make use of concurrency on Python and Kubernetes. We will run some experiments and generate statistics for some combats. Later we can transform these results into actions. We will use some of the Warhammer 40k faction stats to simulate combats and (lots of) dice rolling.

Concurrency vs Parallelism

Concurrency is not the same as parallelism. Parallelism is when two or more tasks are running independently. This is achieved in Python using threads. But concurrency involves multiple tasks using shared resources without mutual blocking. For example, let us think of a server. Instead of pausing for a single database query, it can switch to other requests while the query runs in the background. Doing so, allows for lower response times. Notice the server is not running both requests at the same time. Instead, it jumps to next request once the first request is awaiting completion. Asyncio is the library implementing the concurrency on Python, and we will use it on our experiments.

Creating the combat simulator

Our experiments will consist of combat simulations between Warhammer units. More precisely, we will create a micro service that will simulate combats. Before going deeper, take a look at the system architecture below:

Fig. 1: Simulator Scheme

The first component I created, before the simulator was a PDF parser. It was able to read the indexes from Warhammer (they were open), so a lot of stats are old:

Fig. 2: Processed stats for an aberrant (GSC Army)

The UI shows all the included units, and its different weapons available:

Fig. 3: Selecting attacking and defending units.

Then, I created a simulator for the damage dealt to a unit. If you need to know the rules for a Warhammer combat, you can check them here. The core of the simulator performs these computations. It pulls the stats from the DB by searching by unit’s name.

The main algorithm for the combat simulation is as follows:

Fig 4. Sequence for a simulated combat.

I tested two versions of the code. On the first one, it will wait while the backend searches on the DB for the information about the unit. And on the second one, it will yield control, so the backend can execute other things.

Code

First we code the combat algorithm showed above, using Python:

for _ in range(attacks):
  dice_roll = random.randint(1, MAX_DICE_RESULT)
  if dice_roll == 1:
      continue
  if dice_roll + bonus >= skill:
      successes += 1
  if dice_roll >= MAX_DICE_RESULT:
      successes += 1
      critical += 1
return successes, critical

When a model attacks, it does it a number of times equal to its attacks attribute (see "A" in Fig. 2). A natural 6 (MAX_DICE_RESULT) is a critical hit that can serve to cause additional attacks or damage. With the number of successes the code returns, it calculates the wounds caused:

if strenght >= 2*toughness:
    difficulty = 2
elif strenght >= toughness:
    difficulty = 3
elif strenght == toughness:
    difficulty = 4
elif 2*strenght <= toughness:
    difficulty = 5
else:
    difficulty = 6
for _ in range(hits):
    dice_roll = random.randint(1, 6)
    if dice_roll == 1:
        continue
    if dice_roll + bonus >= difficulty:
        successes += 1
    if dice_roll >= MAX_DICE_RESULT:
        successes += 1
        critical += 1
return successes, critical

There are a few different rules to compute the number of wounds an attack can do. A feeble attack (low strength) should not cause much damage against and a tough target (and the opposite). Notice the sequence of different cases representing this comparison.

Once we had the code, we used Flask to create a microservice that performs the computations. Flask code is outside the scope of this post, so I will only focus on the relevant bits of code:

wounds_array, damage_array, time_array, saves_array = rules.seq_simulator(
        json.loads(data['selected_attack']),
        json.loads(data['selected_defender']),
        logging,
        db)
simulator = sim.Simulator(db)
_, _, p_time_array, _ = await simulator.simulation(
    json.loads(data['selected_attack']),
    json.loads(data['selected_defender']),
    logging)

Notice on the piece of code above how we call for the simulator first (seq_simulator, for sequential simulator) as a regular function, but the second time we call it, we use the keyword await. This keyword will allow the server to know where to yield control over the other processes. Note that this is not a parallel run, but instead the backend server will left this process up to this line, and will come to it later.

Some functions had to be duplicated and I added a loop to test it over a lot of runs (each simulation will do up to 100 different combats). This will allow us to really see what difference using concurrency can do.

Fig 4. UI for the combat simulator

Notice how the efficiency of running concurrent requests instead of using everything sequentially.

Fig 5. Time consumed on combat simulations. Number of simulations on X axis.

Planning with combat actions

Using the combat simulator, we can approximate the outcome of a combat scenario. This tool allows us to create a planning problem that simulates combat situations. For instance, if a unit is predicted to lose a combat, we can code the result as a death. Consequently, the planner might avoid engaging units that are likely to defeat its models.

We will utilize the same boarding planning problem discussed in our previous post, but with updates to include additional predicates. One such predicate is alive, which will be negated when a unit faces an opponent that statistically has a higher chance of defeating it. This addition helps refine the planning process by incorporating survival probabilities into decision-making.

Additionally, we introduce a new action called engage. This action represents the actual combat encounter between units. The outcome of engage will be determined by the statistics computed by the calculator. Since we are not considering (yet!) conditional actions or non-deterministic actions, we just consider an action for every combat a unit can win. For example, if we are playing with space marines, and we are playing against genestealer cults (ordered by who wins who on combat):

genestealers hybrid acolytes < space marines < genestealer aberrants

we can create an action for the space marines to engage (and win) over the acolytes, but not to lose with the aberrants. The action is described as follows:

- name: engage
  parameters:
    - who: agent
    - from: pos
    - with: enemy
    - where: pos
  precond:
    at:
      - who
      - from
    situated:
      - with
      - where
    adjacent:
      - from
      - where
    stronger:
      - who
      - with
  effect:
    neg:
      - situated:
        - with
        - where

Notice the new action will require two units (the unit and its enemy), and will require from these units:

1. To be adjacent (predicate situated is equivalent to predicate at for an enemy)
2. The unit to be stronger than its enemy (predicate stronger).

Once the action si resolved, we consider the enemy is removed from play, hence the negation of the situated predicate.

By integrating these new predicates as preconditions, that we can set on the initial state, for example: stronger_agent_1_enemy_2, we aim to enhance the accuracy and strategic depth of our combat simulations, since it will force the planner to choose plans where the unit does not confront stronger units, hence dying.

Experiments

And finally for the experiments, we will play several games, using different initial configurations. We will start with the same layout of terrain we used in our previous posts. And we will use different troops just to test. First we will consider a unit of space marines agains genestealer acolytes and abberrants, as in our last experiment. In this case, the computed solution is:

Solution: ['', 'board_who_agent_1_from_pos_1_1', 
'move_who_agent_1_from_pos_1_1_to_pos_2_1', 
'move_who_agent_1_from_pos_2_1_to_pos_3_1', 
'move_who_agent_1_from_pos_3_1_to_pos_4_1', 
'move_who_agent_1_from_pos_4_1_to_pos_4_2', 
'move_who_agent_1_from_pos_4_2_to_pos_4_3', 
'move_who_agent_1_from_pos_4_3_to_pos_4_4', 
'move_who_agent_1_from_pos_5_3_to_pos_5_4', 
'move_who_agent_1_from_pos_5_4_to_pos_5_5', 
'engage_who_agent_1_where_pos_5_5_with_enemy_2_what_objective_1', 
'grab_who_agent_1_where_pos_5_5_what_objective_1', 
'move_who_agent_1_from_pos_5_5_to_pos_4_5', 
'move_who_agent_1_from_pos_4_5_to_pos_3_5', 
'move_who_agent_1_from_pos_3_5_to_pos_2_5', 
'move_who_agent_1_from_pos_2_5_to_pos_2_4', 
'move_who_agent_1_from_pos_2_4_to_pos_2_3', 
'move_who_agent_1_from_pos_2_3_to_pos_2_2', 
'move_who_agent_1_from_pos_3_1_to_pos_2_1', 
'move_who_agent_1_from_pos_2_1_to_pos_1_1', 
'secure_who_agent_1_where_pos_1_1_what_objective_1']

Notice how it ignores the aberrants and it focus on attacking the acolytes. Since we have no action for "losing", the planner assumes there is no penalty on crossing over an enemy unit. We can add a predicate called free, indicating if the position to which the unit wants to move is free or occupied, denying the possibility of moving to it in this case. If we do so, notice how the plan changes (it has to start from another corner of the map):

Fig. 6: Notice the space marines forced to board the game from the south instead of north, due to the presence of the stronger Aberrants.

Solution: ['', 'board_who_agent_1_from_pos_1_10', 
'move_who_agent_1_from_pos_1_10_to_pos_1_9', 
'move_who_agent_1_from_pos_1_9_to_pos_1_8', 
'move_who_agent_1_from_pos_1_8_to_pos_2_8', 
'move_who_agent_1_from_pos_2_8_to_pos_3_8', 
'move_who_agent_1_from_pos_3_8_to_pos_4_8', 
'move_who_agent_1_from_pos_3_7_to_pos_4_7', 
'move_who_agent_1_from_pos_3_6_to_pos_4_6', 
'move_who_agent_1_from_pos_3_5_to_pos_4_5', 
'move_who_agent_1_from_pos_4_5_to_pos_5_5', 
'engage_who_agent_1_where_pos_5_5_with_enemy_2_what_objective_1', 
'grab_who_agent_1_where_pos_5_5_what_objective_1', 
'move_who_agent_1_from_pos_5_5_to_pos_4_5', 
'move_who_agent_1_from_pos_4_5_to_pos_4_6', 
'move_who_agent_1_from_pos_4_6_to_pos_4_7', 
'move_who_agent_1_from_pos_4_7_to_pos_4_8', 
'move_who_agent_1_from_pos_3_7_to_pos_3_8', 
'move_who_agent_1_from_pos_2_7_to_pos_2_8', 
'move_who_agent_1_from_pos_2_8_to_pos_1_8', 
'move_who_agent_1_from_pos_1_8_to_pos_1_9', 
'move_who_agent_1_from_pos_1_9_to_pos_1_10', 
'secure_who_agent_1_where_pos_1_10_what_objective_1']

For our second experiment, we switch the position of the units, so the aberrants get to hold the objective. In this case, the planner is unable to come up with a solution, because the space marines cannot win the combat against the aberrants:

Error: solution not found.

Conclusions

In this post, we have done several things. First, we have created a combat calculator, that will provide us with statistical simulation for combats in the game. With the help of this combat simulator, we showed how performance can be increased when using concurrency.

We integrated this combat calculator to our planner, and tested it to see how the planner reacts when it has more actions available. Our planning games simulator is now better, since it allows us to simulate a lot more of scenarios and it reacts better to new information from the environment.

Looking ahead, future enhancements may include AI-driven scenario generation, and also integrate a planner for the enemy units so they can also react to the player moves.

Thanks!