Teaching NPCs to Farm: Reinforcement Learning in Godot

If you’ve ever sunk hundreds of hours into games like Stardew Valley or Animal Crossing, you know the loop is incredibly satisfying: you till the soil, plant seeds, water them, wait, and harvest. But despite their charm, these games often share a common technical limitation: Non-Player Characters (NPCs) operate on rigid, deterministic schedules.

They wake up at 8:00 AM, stand by the river at 10:00 AM, and go to sleep at 9:00 PM. While this creates a nice illusion of life, it’s fundamentally static. I wanted to see if I could push this further. What if the NPCs in a farming sim didn’t just follow a script, but actually learned from your playstyle and adapted to your farm’s needs?

To test this out, I built Minifundium, an isometric 2D farming simulation prototype in Godot Engine 4. My goal was to create a sandbox to experiment with Reactive Artificial Intelligence (Reactive AI) and Reinforcement Learning (RL) - without relying on heavy Python wrappers, external sockets, or clunky Machine Learning (ML) agents. I wanted it all running natively in lightweight GDScript.

Here is how I taught a digital chicken how to automate a farm.

Setting the Stage: Component-Driven Architecture

Before writing any complex Artificial Intelligence (AI), the game needed a solid architecture. Tightly coupling an NPCs movement code with its decision-making logic is a quick way to create a spaghetti codebase.

I relied heavily on Godot 4’s node system to implement the Strategy Pattern. Basically, I split the NPC into two distinct parts:

1. The Controller: a physical body that handles 2D Navigation, plays animations, and interacts with the world.
2. The Brain: an interchangeable child node that processes the environment and tells the controller what to do.

This decoupling was a lifesaver. It allowed me to hot-swap different “brains” into the exact same chicken sprite to test different AI behaviors on the fly.

To keep interactions clean, everything in Minifundium is component-driven. For example, my trees only register damage if they overlap with a Hitbox where current_tool == DataTypes.Tool.AXE. The world itself is built using Godot 4’s new TileMapLayernodes, separating Water, Grass, Ground, and Tilled Dirt into manageable grids.

Leveling Up the AI: From State Machines to Q-Learning

To make sure the system actually worked, I built the AI up in four distinct iterations (Laboratories).

Laboratory #1: The Puppet (Commanded Agent)

First, I made the chicken act as a simple puppet. It stood in an Idle state until I hovered over it and pressed the R keyto issue a manual harvest command. It would navigate to the crop, play the animation, and go back to sleep. This proved the physical controller worked perfectly.

Laboratory #2: The Finite State Machine (The Industry Standard)

Next, I gave it a classic Finite State Machine (FSM). Driven by a timer, the brain would periodically query the environment. If it found a mature crop, it harvested it. If the map was empty, it wandered randomly. This is how most indie games handle NPCs. It works, but it’s inflexible.

Laboratory #3: Native Reinforcement Learning (Tabular Q-Learning)

Here is where things got interesting. I wanted the AI to learn, but I didn’t want to import heavy machine learning frameworks. Instead, I built a lightweight, tabular Q-Learning analogue purely in GDScript.

While traditional Q-Learning relies on environment rewards, my analogue uses a player-driven imitation approach to update the matrix. I gave the brain a 2x2 Transition Matrix - essentially a dictionary mapping current conditions to historical selection weights. Initially, the paths for Idle vs. Harvestare perfectly balanced:

var matrix = {
  "Idle": {"Idle": 1, "Harvest": 1},
  "Harvest": {"Idle": 1, "Harvest": 1}
}

How it learns: Every time I manually commanded the chicken to harvest (pressing the R key), the brain intercepted the event and updated its matrix, adding a +1weight to the priority of that selection inside its row:

func on_player_command() -> void:
  var crop = chicken.find_ready_crop()
  if crop:
    matrix[last_state]["Harvest"] += 1

    chicken.target_crop = crop
    last_state = "Harvest"
    chicken.state.transition_to("Movement")

How it acts: When an action ends and the chicken needs to decide what to do next autonomously, it doesn’t use a hardcoded if/elsestatement. It queries its last_staterow and executes a probabilistic roulette wheel selection:

func _choose_weighted_action() -> String:
  var choices = matrix[last_state]
  var total_weight = 0

  for w in choices.values():
    total_weight += w

  var random_val = randi() % total_weight
  var current_sum = 0

  for state in choices.keys():
    current_sum += choices[state]
    if random_val < current_sum:
      return state

  return "Idle"

Instead of executing a fixed routine, the chicken begins to statistically prioritize the tasks I ordered it to do most often. It starts developing its own “work personality” based on my specific playstyle.

Laboratory #4: The Hive Mind (Group Learning)

For the final test, I wanted to see what would happen if I scaled this up to multiple agents. Using Godot’s Autoloadsystem, I created a GlobalAisingleton and moved the transition matrix from the individual chicken to this shared global node.

The results were amazing, but a bit chaotic. When I trained Chicken Ato prioritize harvesting, the probabilistic weights updated globally. When Chicken Band Chicken Cevaluated their next moves, they read from this shared knowledge base. I had accidentally created a farming collective consciousness: teaching a single agent instantly optimized the productivity of the entire flock.

The Inevitable Bugs: Infinite Loops

Implementing RL in a real-time game loop is rarely goes smoothly. The biggest hurdle I ran into was weight saturation.

If I mass-spammed the R keyto train the chickens quickly, the weights in the global matrix increased sharply. This caused the roulette algorithm to skew so heavily that the chickens got stuck in an infinite loop, constantly trying to harvest empty plots of land and ignoring everything else.

To fix this, I engineered a mathematical clamping function directly into the matrix to normalize the data and cap the maximum weight.

But more importantly, I gave the AI a reality check. I added a predictive context validator. Before the controller executes an action chosen by the AI’s roulette wheel, it checks the actual game world: “Is there actually a crop ready to harvest right now?”. If the answer is no, the weight for that action is temporarily penalized to zero, forcing the AI to spin the roulette wheel again and pick a viable alternative. This completely eradicated logical deadlocks.

The Takeaway

Building Minifundium showed me that we don’t need massive, resource-heavy neural networks or clunky external APIs to make game worlds feel alive.

By leveraging clean component-driven architecture and simple probabilistic matrices natively in GDScript, we can move away from static schedules. Giving players NPCs that actually learn from their inputs makes the game world feel incredibly organic, and it is entirely viable for solo devs and indie teams to implement right in the engine.

[embed]GitHub - llerandi/minifundium: A farming simulation videogame for researching NPC Artificial… A farming simulation videogame for researching NPC Artificial Intelligence. Play as a rabbit and interact with a…github.com