Why I trained 80 cars to race using nothing but mutation and selection

A short field report from building [f1-neuroevolution]: what gradient-free learning gets you, what it costs, and where the cars surprised me.

I spent a few weekends building a thing I’d wanted to build for years: a browser-native simulation where a population of dumb agents learns to drive race cars on a track. No reward functions hand-tuned to the third decimal. No PPO. No replay buffer. Just mutation and selection — neuroevolution, the algorithm that’s been quietly underrated since deep RL became fashionable.

The result is up on [github.com/aifriend/f1-neuroevolution]. Eighty cars per generation, seven tracks of increasing nastiness, a plateau detector that decides when the population is “ready” to face the next level. You open the page, you watch them get less terrible.

This post is what I learned, both from the cars and about the algorithm.

What gradient-free learning is actually good at

The pitch for reinforcement learning is mathematical elegance: you have a reward signal, a policy gradient, and a stable theoretical story for why your loss should go down. The pitch for neuroevolution is closer to “I don’t want to think about backprop today.” That sounds like a weakness. It turns out to be a feature.

Three properties matter.

You don’t need differentiability. The fitness function can include collisions, lap counts, lateral acceleration spikes — anything you can measure. You don’t have to wrap them in something the chain rule can love. In f1-neuroevolution, fitness is distance_travelled × average_speed — 100 × collision_count. A clean expression of “go fast, don’t crash.” Nothing differentiable about the second term, doesn’t matter.

You don’t need an environment that backprops. This was the unlock for browser-native AI. The simulator can be a 2D-ish physics layer written in plain JavaScript. No PyTorch on the client. No WebGL compute shaders for backward passes. The cars evaluate a neural network forward — that’s a tight loop of vector multiplies that even slow JavaScript handles fine for 80 small networks.

You can use selection pressure as a curriculum. This is the property I underestimated. When a track is too hard, the population’s best fitness stops improving — every car dies in the same corner. That’s not a hyperparameter to tune; it’s a signal that the task is currently impossible and the next generation will inherit the same impossibility. So I built a plateau detector. When best-of-generation fitness stays flat for 20 generations, the whole population moves to a slightly harder track. Cars that learned to corner now have to learn to corner faster. Some die; the ones that survive become the substrate for the next adaptation.

That last part is the most interesting thing in the project. I never told any car what skill to learn next. I just kept changing the environment when they stopped struggling.

What it costs

Three honest weaknesses I bumped into.

Sample inefficiency is real. RL papers will tell you that PPO needs 1⁰⁶ environment steps to crack a halfway-decent control problem. Neuroevolution needs more. My 80-car population takes ~3,000 generations to clear all 7 levels — call it 240,000 episodes. A well-tuned PPO would probably get there in half the wall-clock time on the same hardware.

No credit assignment. When a single car has a great run, you can’t tell whether it was its steering policy or its throttle policy that made the difference. Both genes get carried into the next generation together. RL with policy gradients can decompose that more cleanly.

Diversity collapse. A few generations in, if one genome is meaningfully better than the rest, it dominates the next generation, and then the next. Variance shrinks, mutation has to work harder to break through plateaus, the curriculum stalls. Elitism + larger mutation sigma + occasional injection of fresh random genomes — these are all dials I had to tune to keep the population alive.

The cars that surprised me

Two moments where the agents did something I hadn’t predicted.

Cornering with their bumper. On track 4 (hairpins), the best car early on learned to take the corner by kissing the inside wall. Its sensor array gave it distances to walls; the inside wall on a hairpin gave the strongest signal; the easiest policy was “stay glued to it.” When I increased the collision penalty, that strategy disappeared and the cars learned to brake-and-rotate. The selection pressure was always there. I just gave it a different shape.

A new strategy on track 6, after I added wind. I added a small lateral force to track 6 — wind gusts, basically. Within 200 generations, several genomes had independently evolved a “drift compensation” behaviour: their steering was permanently biased about 3 degrees into the wind. Note that there’s no wind-sensor in the input. The cars were predicting the wind from track position because the gusts were correlated with the second long straight. That’s the neat part of evolutionary search — when the environment has structure, the population will find it, even if the structure isn’t represented in the sensors.

What this means for “real” AI

I don’t think neuroevolution is going to replace deep RL. The sample-inefficiency problem is fundamental. But I think the recent corner of the field interested in open-ended evolution and curriculum-as-selection-pressure is one of the more underrated research directions. POET, ECO, and the various pop-curriculum lineages share the same intuition: the environment is the policy, in a way. Change it as fast as the agents can learn, and they keep getting better. Stop changing it, and they plateau.

There’s also something neurally interesting here. Real brains don’t backprop. They get one local error signal — at best — and a lifetime of selection pressure across populations and across generations. F1-neuroevolution is a toy, but it’s a toy that runs on the same algorithm life does. That feels worth keeping around.

What’s next

I’m pulling the project in two directions.

The first is more sophisticated: add NEAT-style structural mutation so the topology of the network can evolve, not just the weights. This is what makes NEAT and HyperNEAT genuinely interesting — bigger networks emerge when bigger networks are actually needed.

The second is more practical: make a comparison mode where you load two trained genomes and watch them race head-to-head on the same track. Same idea as Strava segments. Different genome lineages, same track, who finishes first. I want to make a leaderboard.

If any of this is interesting, the code is on GitHub and the issues tab is open.

neuroevolution, reinforcement-learning, ai, genetic-algorithms, machine-learning

— -

Jose Lopez is an AI engineer based in Madrid working on the intersection of biological and artificial intelligence — find more at [github.com/aifriend] or [auto-latam.com].