Moving Through the World: Teaching Robots to Say “I Don’t Know” — And Why That’s Only the Beginning

Author: Berend Watchus. Independent AI & Cybersecurity Researcher. Publication: OSINT Team, online magazine

Moving Through the World: Teaching Robots to Say “I Don’t Know” — And Why That’s Only the Beginning

copyright: https://arxiv.org/pdf/2606.04853

[embed]Teaching Robots to Say 'I Don't Know' : SENTINEL for Uncertainty-Aware SLAM Low-cost 2D LiDARs lack the intensity channel that higher-end sensors use to diagnose measurement failures, yet they…arxiv.org

The core problem is movement in physical space

A robot moving through physical space shares that space with other agents. Those agents move unpredictably, pursue goals the robot cannot observe directly, and sometimes operate on entirely different assumptions about what is happening around them. This is not an edge case. It is the normal condition of physical space everywhere humans live and move.

A new paper from BuildMachineLabs, appearing on arXiv this week, addresses one precise slice of this problem honestly and well. SENTINEL gives cheap range-only LiDAR sensors a diagnostic channel they have never had. When the sensor is lying — because a laser beam passed through glass or bounced off a mirror — SENTINEL detects the corruption, suppresses the bad data, and falls back to wheel odometry. The robot says: I do not know where I am. It does not confidently build a wrong map.

Tested on real hardware in a 185cm × 245cm arena across glass, mirror, and reflective paper conditions, the system produces a 3.8× separation in reliability scores between clean and glass conditions. Critically, none of these failure modes appear in standard Gazebo simulation, which is itself an important finding: the physical world does things that simulated worlds do not, and a robot that has only ever moved through simulation has not learned to move through physical space.

This is a real contribution. It is also one layer of a problem with many layers above it. To see those layers clearly, it helps to watch how humans actually move through physical space with other agents present.

What human movement in shared space actually involves

[embed]The Body the AI Never Had: Why Self-Driving Systems Keep Failing at the Obvious The Body the AI Never Had: Why Self-Driving Systems Keep Failing at the Obvious Author: Berend Watchus Independent AI &…osintteam.blog

In The Body the AI Never Had (OSINT Team, April 2026) I introduced a taxonomy of six parallel heuristic channels that human road users run simultaneously below conscious analysis when moving through shared physical space.

A single encounter illustrates all six. Cycling on a rural path in the Arnhem area, I encountered a parent with a very young child on a balance bike. I slowed to near-zero before the child had done anything dangerous. Not because trajectory data indicated danger. Because the scene instantly loaded a complete behavioral model: toddler on balance bike, no traffic awareness, attention goes wherever interest goes, body follows attention, trajectory is therefore effectively random.

Then the parent pointed left. Toward horses in a field fifty meters away. The child’s head turned. The body drifted diagonally right — not intentionally, not toward the horses, but as a pure mechanical consequence of the head turning. The child had not decided to enter my lane. The child was simply no longer a road user. Cognitively they were already in the field with the horses. In a baby voice addressed to animals fifty meters away: “Hi paardjes.” (=hi there horsies)

What I ran in that two-second encounter was not a classification pipeline. It was six parallel channels: silhouette recognition triggering precautionary response, damage potential assessment, trajectory geometry, a model of what cognitive world the other agent was inhabiting, calibrated uncertainty about what might happen next, and social contract reading. None of these required clean classification or high confidence scores. They fired on rough heuristics and context.

SENTINEL implements Channel 5 — calibrated uncertainty — for sensor physics. The child’s position would have registered accurately in its sensors. Reliability score high. Data clean. The interpretation of what that position meant required something the architecture does not possess: a model of which world the other agent was currently living in.

Physical space is collaborative logistics

[embed]Not a war game, not a trolley problem: traffic in space and on earth is collaborative logistics Author: Author: Berend Watchus, Independent AI & Cybersecurity Researchermedium.com

In Not a War Game, Not a Trolley Problem (OSINT Team, May 2026) I argued that movement through shared physical space — road, rail, maritime, aviation, space — is fundamentally collaborative logistics. The entire architecture of signs, signals, lane markings, and right of way rules rests on one foundational assumption: everyone present has accepted, even implicitly, the basic terms of the shared movement game.

The evidence for how well this works when the assumption holds is striking. In 2024 the aviation industry carried five billion passengers on over forty million flights. Against that volume there were seven fatal accidents. Nearly half a century has passed between mid-air collisions in US airspace. This is what mature collaborative traffic management produces — not zero accidents, but accidents so rare that decades pass between events of a given type. The same pattern appears in rail, maritime, and road transport wherever coordination infrastructure has been allowed to mature.

The autonomous systems field would do well to study this before reinventing it. A Hamilton-Jacobi reachability paper I examined in that article modeled satellite collision avoidance as a zero-sum adversarial game — worst-case kamikaze steering by the other satellite. In seventy years of spaceflight, satellite-on-satellite kinetic attack has never occurred. Not once. The paper brought battle axes to a carpentry job. The correct tool is the dispatcher manual that every other transport domain wrote generations ago.

The roundabout near an ambulance illustrates the collaborative system at its most vivid. Twenty drivers, no choreography, no instructions beyond the sound of a siren, collectively improvise a solution that gets the ambulance through while nobody collides. Each driver simultaneously locates the sound, assesses their own position relative to the ambulance’s likely path, reads what the nineteen other drivers around them are about to do, and improvises a position that serves the collective solution without any central coordination. This is distributed real-time collaborative awareness at the level of ordinary drivers. The autonomous vehicle that cannot participate in this collective reorganization is missing the baseline that average human drivers demonstrate routinely.

When the collaborative assumption breaks down

The collaborative model has an outer boundary. The authorities designing traffic infrastructure prefer participants to play the collaborative movement game. The system was built for willing participants. When participation cannot be assumed, the engineering runs out.

This happens in ways that range from the innocent to the catastrophic, and understanding that range is essential for autonomous systems designed to move through physical space where all of these situations occur.

The toddler greeting horses has exited the collaborative movement game through joy and magical thinking, with no awareness of the exit. The carnival crowd in the southern Netherlands in February, where costumes defeat silhouette recognition and people ignore lanes, represents a collective temporary exit from normal movement rules that every local driver accommodates instinctively by dropping to walking pace — what I called carnival mode in The Body the AI Never Had. The autonomous system has no calendar, no cultural memory, no ability to recognize that the same physical road requires completely different behavioral parameters depending on the day of the year.

A person in psychiatric crisis at a rail crossing has not exited the collaborative frame through inattention or joy. They have reached a point where their own continuation has ceased to feel viable, and are using the traffic infrastructure instrumentally — with precise awareness of its physics — as a means toward an end the system was never designed to address. The crisis signs now appearing at Dutch rail crossings — carrying a phone number (suicide hotline) and the words you are not alone — represent the infrastructure expanding its communicative function beyond movement regulation to address the mental state of a specific category of agent present at that location. The engineering has no answer for this situation. The human connection might.

The terrorist driving a vehicle into a crowd represents deliberate weaponization of the collaborative movement system against its own participants. The attack works precisely because every pedestrian and driver is operating on the assumption of shared rules. That assumption is the attack vector.

Where game theory becomes the right analytical tool

These situations — the toddler, the carnival, the crisis, the terrorist — share a structure that standard approaches to autonomous navigation cannot address, because they all involve agents operating on different models of what is happening in the shared physical space.

This is where hyper game theory, developed by Peter Bennett in 1980, becomes the correct analytical framework. Standard game theory assumes all players share the same awareness of the game they are in, its rules, their role, and its objectives. Hyper game theory models the more common reality: players do not share that awareness. The asymmetry can take many forms and does not require malice or conflict to exist.

The hunter using fake mating calls knows they are in a hunting game. The deer believes it is in a mating game. One physical space, two incompatible game models, only one player aware of both. The manipulation in a romantic relationship where one partner is simultaneously playing the love game and a control game runs the same structure. An office where employees are optimizing performance in a game the boardroom has already decided to discontinue — through acquisition, sale, or strategic pivot — is a hyper game structure that emerges from information asymmetry and hierarchy, not from bad faith. The tech person who has inadvertent visibility through system access that nobody intended them to have occupies a different information layer again.

It is also worth noting that complete game awareness may not be available to any participant. If what participants take to be reality is itself a layer of representation — in Baudrillardian terms, a simulacrum with no original referent — then all players are operating in a game whose ground rules none of them can fully access. This is not merely philosophical. It establishes that autonomous systems should be designed for irreducible uncertainty about the full structure of the situation they are navigating, not for a world that can eventually be fully mapped and classified.

For moving autonomous agents in physical space, the practical implication is this: the toddler is in the Disney animal friends game while physically inside the traffic game. The person in crisis is using the movement infrastructure as an instrument in a game of their own. The terrorist is in a warfare game inside a system every other participant believes is collaborative logistics. A hacked autonomous vehicle’s passengers believe they are in the collaborative movement game while a remote threat actor is playing a cyber warfare game through the vehicle’s own systems. In every case the autonomous system, operating on the assumption of one shared collaborative game, cannot identify the mismatch.

Teaching autonomous systems to identify which game each agent in their physical environment is currently playing — and to recognize when a game other than the collaborative default is operating — is the deeper unsolved problem that SENTINEL’s sensor reliability contribution begins to make addressable, one foundational layer at a time.

The spectrum of movement capability in physical space

The full range of what moving autonomously through shared physical space requires can be mapped against trained human benchmarks, from the collaborative ceiling to the tactical edge.

The ambulance driver is the ceiling of collaborative movement mastery. They protect a life inside the vehicle while protecting every life outside it simultaneously. They read who has heard the siren and who hasn’t, who will freeze and who will move correctly, who is about to panic-brake instead of pulling over. They are the agent that twenty ordinary drivers collectively reorganize around at a busy roundabout, each one updating their behavior because they have correctly identified that a new game — ambulance priority — has been requested of the shared space. The collaborative movement game played at its absolute ceiling. This is the civilian benchmark no current autonomous system meets.

The close protection driver adds a parallel tactical layer to the same physical movement task: threat geometry assessment, extraction planning, transition between collaborative and defensive modes based on real-time identification of which game is actually operating in the shared space.

Leo Prinsloo, the South African cash-in-transit driver whose 2021 armed ambush evasion was captured on dashcam and seen worldwide, represents the outer edge of what trained humans execute in physical space under fire. During that ambush he briefly transitioned from defensive extraction to offensive use of the vehicle — attempting to steer into armed attackers on foot. He did not succeed but he executed the transition. That decision required identifying which game was actually operating, assessing the priority of what the vehicle was carrying, calculating that offensive action improved outcomes, and executing it with the same physical capabilities — spatial awareness, navigation, throttle, steering, braking — that every autonomous vehicle already requires for routine movement through space.

This capability is not pretty. In 99.999% of vehicles it is completely irrelevant. For SOF operators, intelligence services, and high-value asset protection it is a legitimate and specific design requirement. And it is precisely here that the trolley problem and the war game framework — which I dismissed in Not a War Game, Not a Trolley Problem as catastrophically misapplied defaults for routine traffic — become genuinely appropriate tools. Not as defaults. As the correct frameworks for the narrow moment when the collaborative infrastructure has been deliberately destroyed and a forced choice between bad outcomes exists in physical space. The trolley problem is not a fantasy. It is what deliberate adversarial action forces into existence at the outer edge of the spectrum.

The foundational stack is identical across the entire spectrum

This is the connection that gives SENTINEL its full significance.

Whether the mission is delivering groceries, getting a wounded person to hospital, extracting a VIP, or evading an armed ambush, the vehicle still needs spatial awareness of the environment in real time, navigation that adapts to a changing picture, throttle control precise enough for the situation, steering accurate enough to place the vehicle where the mission requires, and braking calibrated to what the moment demands.

The decision architecture changes across the spectrum. The physical movement stack does not. SENTINEL’s sensor reliability contribution — teaching the robot to say I do not know about a laser beam hitting glass — is a prerequisite for every layer above it. A system that cannot trust its own sensor data cannot navigate a carpark reliably, let alone identify counter-collaborative behavior in shared physical space, let alone execute a tactical exit, let alone make the judgment call Prinsloo made under fire.

What remains

SENTINEL is an honest, hardware-validated step toward a robot that knows its own limits in physical space. That is rarer than it should be.

But one channel, partially implemented, on a $575 sensor stack in a 185cm arena, leaves five channels untouched. It leaves the ambulance driver’s collaborative game awareness unbuilt. It leaves the collective roundabout reorganization the system cannot participate in. It leaves the hyper game inference architecture — the capacity to identify which game each agent is actually playing in shared physical space — entirely undesigned.

The robots are learning to say I do not know about a laser beam hitting glass. The longer road is teaching them to move through a world where a toddler is greeting horses, a carnival is in progress, twenty drivers are reorganizing around a siren, a threat actor has silently changed the game, and a trained professional is making a judgment call that no classification pipeline has ever been asked to make.

That world is not a test environment. It is everywhere humans actually live and move.

Which is where the robots need to go.

References:

Watchus, B. The Body the AI Never Had. OSINT Team, April 2026.

Watchus, B. Not a War Game, Not a Trolley Problem. OSINT Team, May 2026.

BuildMachineLabs. Teaching Robots to Say “I Don’t Know”: SENTINEL for Uncertainty-Aware SLAM. arXiv:2606.04853, June 2026.

Bennett, P. Hypergames: Developing a Model of Conflict. Futures, 1980.