MissionOS (AI-agent-based mission control) Under Strong Wind: A Recovery Experiment

Tokyo Station → Akihabara, PX4/Gazebo SITL, 1kg payload, 9m/s wind

This is not a story about a successful delivery.

In the previous MissionOS experiment, I wrote about taking a natural language delivery request and completing a simulated delivery mission on PX4/Gazebo SITL. MissionOS generated the mission, executed it in the simulator, released the payload, and verified the result.

This time, I wanted to test something less glamorous, but more important.

What happens when the mission does not go well?

I ran a simulated delivery mission from Tokyo Station to Akihabara Station with the following conditions:

• Start: Tokyo Station
• Destination: Akihabara Station
• Payload: 1kg
• Wind: 9m/s
• Runtime: PX4/Gazebo SITL

SITL means Software-In-The-Loop. Instead of flying a real drone, PX4 runs as the flight controller software, while Gazebo simulates the world around it.

The result was simple:

The drone did not reach the destination. The payload was not delivered. The mission failed.

But for MissionOS, this was still meaningful progress.

The important part was not that the AI completed the mission. It did not. The important part was that MissionOS handled the failure without pretending it was a success.

The Recovery Agent proposed Return To Launch. A human operator approved it. The command was accepted by the active runner. But MissionOS did not observe actual return progress.

So MissionOS did not call it a successful return.

That distinction is the core of this experiment.

What MissionOS is trying to do

MissionOS is not a system where an AI directly flies a drone.

The closer metaphor is mission control.

PX4 handles low-level flight control: attitude, motors, navigation modes, and vehicle behavior. MissionOS sits above that layer. It creates missions, checks whether execution is allowed, monitors runtime state, proposes recovery actions, requires human approval, dispatches approved commands, and verifies what actually happened.

The roles are intentionally separated:

• AI evaluates the situation and proposes actions.
• The human operator approves or rejects those actions.
• Policy and safety rules constrain what is allowed.
• The executor only sends approved commands.
• The verifier records what was actually observed.

That separation matters.

An AI saying “Return To Launch is recommended” is not the same thing as an AI sending an RTL command. A command being accepted is not the same thing as the vehicle actually returning. A map showing a route is not the same thing as proof of delivery.

MissionOS is designed to avoid mixing those facts.

From natural language to SITL execution

The mission started from a natural language request:

Run a 1kg payload mission from Tokyo Station to Akihabara Station under 9m/s wind.

MissionOS did not immediately execute it.

The flow was deliberately staged:

missionos say
missionos approve
missionos run
missionos start-sitl
missionos execute-sitl --live-flight
missionos rtl
missionos map

At first glance, this may look overly careful.

But the separation is the safety model.

“Proposed,” “approved,” “prepared,” “started,” “executed,” “recovery requested,” and “verified” are different states. MissionOS keeps them separate so that the system does not accidentally turn a suggestion into an action, or an accepted command into a success claim.

CLI flow — The terminal UI is in Japanese, but the important flow is: natural language request → mission proposal → human approval → SITL startup → live execution. MissionOS intentionally separates proposal, approval, execution, and verification.

The staged missionos chat flow (Japanese UI). A natural-language request becomes a bounded proposal (wind=9.0m/s, payload=1.0kg), then moves through approve → prepare → start SITL → live execution as separate states. The live lines show battery falling 98.8% → 66.1% under 9m/s wind, ending in proposal_guardrail_passed:return_to_launch with risk=battery_projected_insufficient_for_route, route_deviation_above_limit.

What happened during the mission

Once live SITL execution started, the mission quickly entered a difficult state.

Progress toward the route did not grow as expected. Terrain clearance was also insufficient. In this context, terrain clearance means the margin between the vehicle and the ground or terrain model. The target clearance was around 30m, but the runtime state showed that this margin was not consistently healthy.

The wind condition made the mission even harder.

At this point, the Recovery Agent became relevant.

The Recovery Agent watches runtime telemetry: position, altitude, battery, route progress, terrain clearance, route deviation, and other state signals. It does not directly control the drone. It evaluates the situation and proposes what the operator should do next.

In this run, the agent proposed Return To Launch.

What the Recovery Agent actually judged

It is worth being precise about what “proposed RTL” means here, because this is where the intelligence lives.

The Recovery Agent did not match a threshold to a fixed action. It was given the runtime picture as evidence — route progress that was not growing, terrain clearance sitting under its ~30m target, sustained 9m/s wind, and the resulting route deviation — and asked to judge the situation and choose a bounded response. Its read was that route completion was no longer safely reachable within the mission envelope, but that the vehicle was still in a recoverable state rather than an emergency one.

From there it selected among real alternatives, not a single canned output. Continuing was off the table — both the agent’s read and a guardrail rule treat active runtime risk as a no-continue condition. An immediate land was held in reserve for the case where drift made a return unsafe. Return To Launch was chosen as the bounded response that best fit a recoverable-but-failing mission, and it was emitted as a proposal with the expected outcome attached, not as a command.

Only then did the boundaries close around that judgment. A deterministic guardrail checked that RTL was even admissible — for example, that projected battery was sufficient to return home — before the proposal was allowed to stand. A human operator still had to approve it. The AI owned the judgment; it did not own the authority.

This is the distinction I want to keep visible alongside “ACK is not success.” The honesty about the outcome only matters because the decision in front of it was a real decision. RTL here was the agent’s reasoned choice under wind, bounded by rules and gated by a human — not a lookup.

That proposal was still only a proposal — execution waited on the operator.

Recovery Agent console — The terminal UI is in Japanese. The important part is that the Recovery Agent detects that the mission is no longer progressing safely and recommends missionos rtl. The AI proposes Return To Launch, but the operator must explicitly approve it.

The Runtime Recovery Agent’s operator console — the judgment, in the agent’s own words. It reads that the route can no longer be completed (battery would need ~5.7× what remains), detects battery shortfall, large route deviation, and intermittent telemetry, and notes the vehicle can still return safely (~27% left on arrival at the 120 m home point). It proposes RTL and marks continuing as not recommended: proposal_guardrail_passed; dispatch_authority=False. A proposal, not a command — approval happens in a separate pane.

AI proposes, the human approves

The operator sent RTL with human approval:

missionos rtl --yes

MissionOS then dispatched the request to the active runner.

The runtime record showed that the command was accepted:

dispatch_status=queued_for_active_runner
recovery_action=return_to_launch
runner_ack=accepted
tracking=MAV_CMD_NAV_RETURN_TO_LAUNCH
outcome=return_progress_not_observed

This is the most important part of the experiment.

runner_ack=accepted means the runner accepted the command. It does not mean the drone returned. It does not mean the drone landed. It does not mean the recovery succeeded.

The actual outcome was:

outcome=return_progress_not_observed

After the operator approved RTL, the runner accepted it — runner_ack=accepted, tracking=MAV_CMD_NAV_RETURN_TO_LAUNCH — but the recorded outcome=return_progress_not_observed (home_delta=6m, landed=False). MissionOS keeps delivery/progress/physical claim=false. Accepted is not returned.

In other words, MissionOS observed that the RTL command was accepted, but it did not observe actual return progress.

So MissionOS did not claim return success.

This is the line I care about most:

ACK is not success.

An acknowledgment only means “the command was received” or “the command was accepted by the relevant layer.” It is not proof that the physical or simulated world changed in the expected way.

For physical AI systems, this distinction is critical.

The boundary between command and reality

There are many boundaries that are easy to collapse in demos:

Command requested
Command approved
Command dispatched
Command accepted
Vehicle state changed
Vehicle returned
Vehicle landed
Mission completed

A weak system compresses all of these into one story:

The AI handled it.

MissionOS does not do that.

In this experiment, the system only confirmed part of the chain:

RTL requested
RTL approved
RTL dispatched
Runner accepted it
Return progress was not observed

That is why the result remained a failure.

And that is exactly the point.

A system that touches the physical world must be able to say not only what happened, but also what did not happen, and what cannot yet be claimed.

Runtime Recovery panel — The Japanese UI shows the runtime recovery state. The key English-readable lines are runner_ack=accepted, tracking=MAV_CMD_NAV_RETURN_TO_LAUNCH, and outcome=return_progress_not_observed. MissionOS recorded that the RTL command was accepted, but did not claim that the vehicle actually returned.

The map is not proof either

MissionOS also generated a map view with missionos map.

The map overlays the intended route and observed state on real-world geography using OpenStreetMap. It shows Tokyo Station, the Akihabara direction, the home point, the drop-off target, and the current position.

This is useful.

But it is not proof of delivery.

In this run, the map had only limited telemetry points. It was useful as an evidence display, but not as a verifier for mission completion.

A line on a map is not a delivery. A plotted route is not a successful flight. A current marker is not proof of return.

This is another distinction MissionOS tries to preserve.

The MissionOS 2D map overlays route and observed position on OpenStreetMap — home (H), dropoff (D), and the current marker. Its own header says it plainly: read-only evidence display, not a verifier, dispatch control, or delivery claim. With a single telemetry sample, it shows where the vehicle was, not that the mission succeeded.

Map view — The map view helps visualize the mission state on real-world geography. However, it is an evidence display, not a delivery verifier. A route line on a map does not prove that the mission was completed.

Why I also looked at LLM invocation cost

Another part of this experiment was less visible, but important: the way the Recovery Agent calls the LLM.

During live flight, telemetry changes continuously. Position, altitude, battery, route progress, and clearance can update many times during a mission. Sending every telemetry update to an LLM would be expensive, slow, and operationally unrealistic.

So MissionOS now treats runtime signals differently.

Some events are hard triggers:

• terrain clearance below a threshold
• battery risk approaching a dangerous state
• severe telemetry degradation
• obvious execution failure

Other events are soft triggers:

• route deviation growing
• progress stalling
• wind remaining high
• weak but not yet fatal warning signs

The idea is simple:

• If a hard trigger appears, ask the AI sooner.
• If only soft triggers appear, wait longer.
• If the mission is healthy, do not call the AI.

This avoids calling the LLM on every telemetry frame. And it is worth being clear about what these thresholds do: they decide when to ask the AI, not what it decides. The action is still the agent’s judgment, not the trigger’s.

But the important rule remains the same:

A trigger is not a command.

A trigger is a fact. The AI can evaluate that fact and propose an action, but the action still has to pass through authority and safety boundaries.

Local LLMs are role-dependent

I also tested a local LLM backend through Ollama.

The default MissionOS backend remains Gemini, but for local experiments I tried Gemma 4 26B MoE.

The result was mixed.

For planning, Gemma 4 26B was usable. It could parse the natural language request and help produce route and mission parameters.

But for the live Recovery Agent, it was not suitable in this test.

There were two main problems:

One response took about 133 seconds.
Structured JSON output was not stable enough.

That matters because the Recovery Agent needs timely and structured decisions.

This does not mean “local LLMs do not work.”

It means the question has to be role-specific.

A model that is good enough for planning may not be good enough for runtime recovery. A model that can write a useful paragraph may still fail if the system requires strict JSON, low latency, and guardrail-compatible output.

For this recovery run, Gemini was the practical choice.

Was this experiment a failure?

As a delivery mission, yes.

The drone did not reach Akihabara. The payload was not released. Delivery completion remained false.

This was not a successful delivery.

But as a MissionOS experiment, it was valuable.

MissionOS created a mission from natural language. It recorded human approval as a separate boundary. It prepared and started PX4/Gazebo SITL. It entered live execution. It monitored runtime state. The Recovery Agent proposed RTL. The AI did not execute the command by itself. A human approved the recovery action. The runner accepted the command. And finally, MissionOS recorded that return progress was not observed.

That final part matters most.

The system did not lie.

It did not say:

RTL accepted, therefore return succeeded.

It said:

RTL accepted, but return progress was not observed.

That is the kind of failure record I want.

Toward physical systems

MissionOS is still running in simulation here.

This was not a real-world drone flight. It was PX4/Gazebo SITL.

But the closer a system gets to the physical world, the more careful its language must become.

“Dispatched” and “executed” are different. “ACK accepted” and “vehicle moved” are different. “Disarmed” and “landed” are different. “Map displayed” and “delivery verified” are different. “SITL success” and “real-world success” are different.

The easy path is to connect an AI to a command interface and call it autonomy.

The harder path is to preserve the boundaries between AI proposal, human authority, policy constraints, execution, observation, and verification.

MissionOS is trying to build the second path.

Not “AI did everything.” Not “the demo looked successful.”

But:

What was requested?
What was approved?
What was dispatched?
What was accepted?
What was observed?
What still cannot be claimed?

This strong-wind mission did not deliver the payload.

But it did test whether MissionOS could fail honestly.

And in physical AI systems, honest failure handling is not a side feature.

It is part of the foundation.

Closing

The Tokyo Station to Akihabara mission failed.

Under 9m/s wind, route progress was poor, terrain clearance was insufficient, and the Recovery Agent proposed Return To Launch. The operator approved RTL. The command was accepted by the runner. But MissionOS did not observe actual return progress.

So it did not claim success.

That is the point of this experiment.

MissionOS is an operating layer for keeping AI proposal, human approval, execution, and verification separate.

The goal is not to make the AI look powerful.

The goal is to make the system honest about what happened.

ACK is not success.

And for systems moving toward the physical world, that distinction matters.