Testing, Debugging, and Observability for Universal Microservices

UMA’s approach to portable testing, debugging, and observability

*The Rise of Device-Independent Architecture | Part 11*

A half-repaired radio with tangled wires captures the essence of bringing clarity to distributed UMA services through testing, debugging, and observability.

I still recall my first attempt at debugging a distributed system without adequate observability. It was like chasing shadows. One service functioned flawlessly in my local tests, yet another acted differently in staging, while the production logs seemed entirely disconnected. I spent days guessing the faulty segment, patching blindly, and hoping I was addressing the right issue.

In last week’s post, we discussed how UMA makes services verifiable, signed, attested, and policy-enforced without depending on a central authority. That’s a solid foundation, but verification only confirms that what you intended to run is valid. It doesn’t ensure that what’s actually running behaves as expected over time.

That’s where testing, debugging, and observability come into play. In a UMA world, services aren’t tied to a single environment. They can reside in browsers, mobile devices, edge nodes, or the cloud, each with its own unique characteristics. Without a way to test and inspect them everywhere they run, you’re essentially flying blind. This post is about removing that blindfold.

Why Traditional Approaches Fail in UMA’s World

When you build for a single runtime, your testing and debugging setup is predictable. You have a known stack, a fixed set of logs, and a clear path from “run the test” to “see the result.” But UMA changes that dynamic. A single service can run on a mobile device, inside a browser tab, at the edge, and in the cloud, often simultaneously.

In traditional systems:

• Centralized logging assumes all logs can be shipped to one place. That breaks when some runtimes are offline or constrained (for example, a mobile app in airplane mode).
• Environment-specific tests rely on the exact runtime configuration you have in dev or staging. In UMA, those assumptions fall apart when you cross boundaries between runtimes.
• Monolithic debug pipelines depend on language-specific or VM-specific tooling. UMA services might run in WebAssembly on one device and as a native binary on another, making those tools useless in half the cases.

The pain shows up in real projects. Imagine a service that calculates tax rates:

• In the cloud runtime, it passes all tests.
• In the browser runtime, it fails when network latency spikes.
• On mobile, it silently switches to cached data due to an offline mode quirk.

If your tooling is locked to a single environment, you will miss these issues until a customer reports them.

What UMA forces us to do is think cross-runtime first. Testing and debugging need to be portable, just like the services themselves. That means:

1. Tests must run in every supported runtime without modification.
2. Debugging tools must speak to the runtime through a standard interface, not through environment-specific hacks.
3. Telemetry must follow a unified schema, so you can compare events from different runtimes without translation nightmares.

Here is a very simple pattern for cross-runtime test execution using the same portable contract:

// taxService.test.js
import { runService } from "./runtime.js";
import schema from "./taxService.schema.json";
import assert from "assert";

async function testTaxService(runtime) {
  const service = await runService("tax-service", runtime);
  const result = await service.calculateTax({ amount: 100, region: "CA" });

  assert(schema.output.validate(result), "Output does not match schema");
  assert(result.total >= 0, "Total must be non-negative");
}
await testTaxService("browser");
await testTaxService("edge");
await testTaxService("cloud");

The same file can be executed in different runtimes because:

• The service is loaded through a runtime abstraction (runService) instead of direct imports.
• Validation is done through a shared schema, so expectations are consistent everywhere.

This is where UMA’s approach starts to make testing and debugging less about environments and more about contracts.

UMA’s Built-In Testing Principles

UMA considers testing a top priority during runtime, not just an afterthought. In other words, the same portability guarantees that protect your services also extend to your test suites.

Here are the core principles:

1. Contract-Based Validation Before Deployment Every UMA service carries a machine-readable contract that describes its inputs, outputs, and expected side effects. Tests reference this contract rather than runtime-specific assumptions. This makes it possible to validate a service before it ever runs in production.
2. Cross-Runtime Test Packs Instead of separate test harnesses for each environment, UMA supports test packs that can be executed in any runtime browser, mobile, edge, or cloud — without code changes.
3. Self-Describing Tests Test definitions can embed metadata (name, description, dependencies, required environment variables) so that tooling can select which tests to run depending on runtime capabilities.

Here’s a simple example of a portable test pack that can run anywhere:

// tests/taxServiceTests.js
import schema from "../services/taxService.schema.json";
import assert from "assert";

export const tests = [
  {
    name: "Basic Tax Calculation",
    description: "Should calculate tax correctly for standard region",
    run: async (runtime) => {
      const service = await runtime.load("tax-service");
      const result = await service.calculateTax({ amount: 100, region: "CA" });

      assert(schema.output.validate(result), "Invalid output schema");
      assert(result.total > 100, "Total should include tax");
    }
  },
  {
    name: "Offline Mode Behavior",
    description: "Should use cached data when offline",
    run: async (runtime) => {
      runtime.simulateOffline();
      const service = await runtime.load("tax-service");
      const result = await service.calculateTax({ amount: 50, region: "CA" });

      assert(result.source === "cache", "Expected cached result in offline mode");
    }
  }
];

And here’s a cross-runtime runner that makes no assumptions about where it’s running:

// testRunner.js
import { runtimes } from "./availableRuntimes.js";
import { tests } from "./tests/taxServiceTests.js";

for (const runtimeName of runtimes) {
  const runtime = await import(`./runtimes/${runtimeName}.js`);
  console.log(`Running tests on ${runtimeName}...`);

  for (const test of tests) {
    try {
      await test.run(runtime);
      console.log(`✅ ${test.name}`);
    } catch (err) {
      console.error(`❌ ${test.name}: ${err.message}`);
    }
  }
}

This approach means:

• You write tests once and run them everywhere.
• Failures are tied to contract violations, not runtime quirks.
• Adding a new runtime (like a new browser or Wasmer in the cloud) requires no new test code.

How portable test packs fit into UMA

By making the test runner portable and emphasizing contract-driven tests, UMA transforms testing from an environment-dependent activity into a runtime-agnostic capability.

Debugging Across Runtimes Without Losing Your Mind

In UMA, you cannot rely on a single debugger or a single VM. You need a lightweight, portable way to access any runtime that allows you to examine state, modify inputs, and capture traces without altering service code. The key is a small introspection feature that every runtime provides and that each service can choose to enable.

The introspection contract

Define a tiny, optional interface that services can implement. The runtime wires this to transport specific plumbing.

// contracts/Introspectable.ts
export interface Introspectable {
  getDebugState(): Promise<Record<string, unknown>>;
  setDebugFlag(flag: string, value: boolean): Promise<void>;
  inject(input: Record<string, unknown>): Promise<Record<string, unknown>>;
}

Services that support debugging implement it. Others ignore it.

// services/taxService.ts
import { Introspectable } from "../contracts/Introspectable";

type State = {
  cacheHits: number;
  lastInput?: { amount: number; region: string };
  offline: boolean;
};
export class TaxService implements Introspectable {
  private state: State = { cacheHits: 0, offline: false };
  async calculateTax(input: { amount: number; region: string }) {
    this.state.lastInput = input;
    if (this.state.offline) {
      this.state.cacheHits += 1;
      return { total: input.amount, tax: 0, source: "cache" };
    }
    const tax = input.amount * 0.13;
    return { total: input.amount + tax, tax, source: "live" };
  }
  async getDebugState() { return { ...this.state }; }
  async setDebugFlag(flag: string, value: boolean) {
    if (flag === "offline") this.state.offline = value;
  }
  async inject(input: Record<string, unknown>) {
    return this.calculateTax(input as any);
  }
}

A runtime-neutral debug API

Expose the same debug verbs over different transports. In the browser, use a postMessage bridge or WebSocket. In server or edge use a local TCP or gRPC endpoint. The API surface stays the same.

// runtime/debugApi.ts
export type DebugClient = {
  state(): Promise<any>;
  flag(name: string, value: boolean): Promise<void>;
  inject(payload: any): Promise<any>;
};

export function createDebugClient(locator: { transport: "ws" | "local", url?: string, id?: string }): DebugClient {
  if (locator.transport === "ws") {
    const sock = new WebSocket(locator.url!);
    const call = <T>(method: string, params: any) =>
      new Promise<T>((resolve, reject) => {
        const msg = JSON.stringify({ id: locator.id, method, params });
        sock.send(msg);
        sock.onmessage = e => resolve(JSON.parse(e.data));
        sock.onerror = reject;
      });
    return {
      state: () => call("getDebugState", {}),
      flag: (n, v) => call("setDebugFlag", { flag: n, value: v }),
      inject: (p) => call("inject", p),
    };
  }
  // local in-process adapter for CLI or tests
  const svc = globalThis.__umaLocate(locator.id!) as any;
  return {
    state: () => svc.getDebugState(),
    flag: (n, v) => svc.setDebugFlag(n, v),
    inject: (p) => svc.inject(p),
  };
}

Hands-on workflow

Browser workflow

1. Start the app with the tax service.
2. Open the dev console and run:

import { createDebugClient } from "./runtime/debugApi.js";
const dbg = createDebugClient({ transport: "ws", url: "wss://localhost:4444/debug", id: "tax-service" });

await dbg.state();                               // inspect current counters
await dbg.flag("offline", true);                 // simulate offline
await dbg.inject({ amount: 120, region: "CA" }); // run request without UI
await dbg.state();                               // verify cacheHits increased

You just toggled runtime behavior and validated the branch without touching app code or shipping special builds.

CLI workflow

Same verbs, no browser needed.

node -e "import('./runtime/debugApi.js').then(async m => {
  const dbg = m.createDebugClient({ transport: 'local', id: 'tax-service' });
  console.log(await dbg.state());
  await dbg.flag('offline', true);
  console.log(await dbg.inject({ amount: 75, region: 'CA' }));
  console.log(await dbg.state());
})"

Trace what happened and why

Add a unified, portable trace event for every decision. Emit the same shape everywhere so you can compare across browser, mobile, edge, and cloud.

// telemetry/trace.ts
export type TraceEvent = {
  ts: string;
  service: string;
  event: string;
  context?: Record<string, unknown>;
};

export function emit(event: TraceEvent) {
  const line = JSON.stringify(event);
  if (typeof window !== "undefined") {
    // browser: persist to IndexedDB or postMessage to devtools
    console.log(line);
  } else {
    // server or edge: stdout to JSONL collector
    process.stdout.write(line + "\n");
  }
}
// in taxService
emit({ ts: new Date().toISOString(), service: "tax-service", event: "decision", context: { offline: this.state.offline } });

Now your debugging session produces comparable JSONL across all runtimes. That feeds directly into the observability section later.

Sequence diagram of a live debug session

Guardrails and gotchas

Practical rules that keep this safe and useful.

1. Never expose debug verbs without auth. Gate WebSocket or gRPC with mTLS or a signed dev token.
2. Keep introspection read-only by default. Enable setDebugFlag and inject only in dev or under an explicit allowlist.
3. Cap output and redact sensitive fields in getDebugState. Prefer derived counters over raw PII.
4. Make debug builds and production builds behave the same unless a flag is set. The goal is to observe real behavior, not a special mode.

What can readers do today?

1. Add the Introspectable interface to one service.
2. Wrap a minimal debug API in your runtime that maps to getDebugState, setDebugFlag, and inject.
3. Emit a single TraceEvent on every branch decision. Use JSONL so you can diff across runtimes.
4. Script one CLI session that flips a flag, injects an input, and prints the state before and after. Commit it as a reproducible recipe.

Observability as a First-Class Feature

Testing and debugging address problems before and during development, but observability keeps you confident in production. In UMA, observability isn't just an optional add-on or a collection of environment-specific logging tricks. It is an integral part of the service contract and works across different runtimes, just like the service itself.

Unified Telemetry Schema

Each UMA service can emit telemetry in a single, standardized format, whether it's running in a browser, mobile app, edge device, or cloud environment. This enables easy correlation across different environments without translation issues.

A minimal example of a telemetry schema in JSON Schema format:

{
  "$id": "https://example.com/uma.telemetry.schema.json",
  "type": "object",
  "properties": {
    "ts": { "type": "string", "format": "date-time" },
    "service": { "type": "string" },
    "level": { "type": "string", "enum": ["info", "warn", "error"] },
    "event": { "type": "string" },
    "context": { "type": "object" }
  },
  "required": ["ts", "service", "level", "event"]
}

This guarantees that each event, regardless of runtime, includes a timestamp, service name, severity, and event type, with optional context for additional detail.

Emitting Telemetry Everywhere

A single helper can abstract away the runtime-specific differences:

// telemetry.js
export function emitTelemetry(event) {
  const payload = JSON.stringify(event);
  if (typeof window !== "undefined") {
    // Browser: send to IndexedDB or a collector endpoint
    navigator.sendBeacon("/telemetry", payload);
  } else {
    // Node, Edge, Cloud: stdout for log aggregation
    process.stdout.write(payload + "\n");
  }
}

// usage inside a UMA service
emitTelemetry({
  ts: new Date().toISOString(),
  service: "tax-service",
  level: "info",
  event: "calculation",
  context: { amount: 100, tax: 13 }
});

This makes sure the same emitTelemetry call works across all supported runtimes.

Traceability and Trust Boundaries

In UMA, observability is tied to the same trust boundaries used for verification and policy enforcement. Every event can carry metadata about:

• The contract version in use.
• The runtime ID (browser, mobile, edge, cloud).
• The origin of the service binary (attestation reference).
• This allows you to filter events by trust context, which is critical when investigating cross-runtime inconsistencies or security issues.

Unified Telemetry Flow

This clarifies that all environments provide the same structure to your analysis pipeline.

Action Steps for Readers

1. Define a JSON Schema for telemetry events and make it part of your service contract.
2. Implement a single telemetry emitter that works in all runtimes you support.
3. Include trust metadata in every event so you can correlate and filter meaningfully.
4. Build basic dashboards or CLI tools to query telemetry across runtimes before you invest in complex observability platforms.

The Feedback Loop Testing + Observability → Confidence

Testing identifies known issues before deployment. Debugging helps you resolve problems during development. Observability reveals what is happening in production. The real value arises when you combine all three into a continuous loop.

In UMA, the same contracts that guide service behavior also influence your telemetry. This means that any deviation from the contract can automatically trigger a test, generate alerts, or feed into debugging workflows. You are no longer just collecting logs; you are using runtime behavior to improve the system over time.

Turning Telemetry Into Tests

Every telemetry event could serve as a test case. For instance, if you encounter an unexpected branch in production

{
  "ts": "2025-08-08T14:23:00Z",
  "service": "tax-service",
  "level": "info",
  "event": "calculation",
  "context": { "amount": -50, "tax": -6.5 }
}

That negative amount might be valid in some edge case, or it might be a bug. You can automatically capture it as a new input for your cross-runtime test pack:

import { addTestCase } from "./testRegistry.js";

addTestCase("Negative Amount Case", async (runtime) => {
  const svc = await runtime.load("tax-service");
  const result = await svc.calculateTax({ amount: -50, region: "CA" });
  assert(result.total >= 0, "Total should never be negative");
});

Detecting Drift

UMA services specify their expected dependencies, APIs, and output formats. When telemetry indicates a service making unintended calls or producing output that fails schema validation, it signals that the implementation has deviated from its contract.

Example: An automated check that runs nightly against recent telemetry:

import schema from "./taxService.schema.json";
import { getRecentEvents } from "./telemetryStore.js";

for (const event of await getRecentEvents("tax-service")) {
  if (!schema.output.validate(event.context)) {
    console.error(`Contract violation detected at ${event.ts}`);
  }
}

Auditing Runtime Decisions

Because observability events carry trust metadata (contract version, runtime ID, attestation reference), you can reconstruct exactly what happened, where, and under which policy version. This is critical for compliance and incident response.

Example CLI query:

uma-telemetry query --service tax-service --runtime edge --contract v1.2.3 --since "2025-08-01"

This outputs a filtered view of only the events relevant to a specific runtime and policy context.

Continuous Improvement Loop

The cycle continues: telemetry reveals unexpected behavior, tests reproduce it, debugging resolves it, and the improved service returns to the ecosystem with better guarantees.

Action Steps for Readers

1. Feed production telemetry into your test case registry.
2. Automate schema validation for recent events.
3. Use trust metadata to filter and audit runtime decisions.
4. Treat contract violations as first-class incidents, not just warnings.
5. Close the loop: every anomaly should result in a test or a policy update.

Hands-On Example: A Portable Service You Can Test, Debug, and Observe Anywhere

In this walkthrough, we will build a minimal UMA-style portable service, attach a cross-runtime test pack, expose debug capabilities, and emit telemetry. Then we will run it in two different runtimes, one simulating the browser and the other simulating the cloud, to demonstrate that the loop works.

The Service Contract

First, define the service schema so both tests and telemetry can validate it.

// taxService.schema.json
{
  "$id": "https://example.com/taxService.schema.json",
  "type": "object",
  "properties": {
    "total": { "type": "number" },
    "tax": { "type": "number" },
    "source": { "type": "string" }
  },
  "required": ["total", "tax", "source"]
}

The Service Implementation

// taxService.js
import schema from "./taxService.schema.json" assert { type: "json" };
import { emitTelemetry } from "./telemetry.js";

export class TaxService {
  constructor() {
    this.cacheHits = 0;
    this.offline = false;
  }
  async calculateTax({ amount, region }) {
    let source = "live";
    let tax = amount * 0.13;
    if (this.offline) {
      this.cacheHits += 1;
      source = "cache";
      tax = 0;
    }
    const result = { total: amount + tax, tax, source };
    emitTelemetry({
      ts: new Date().toISOString(),
      service: "tax-service",
      level: "info",
      event: "calculation",
      context: result
    });
    if (!schema.output?.validate?.(result)) {
      emitTelemetry({
        ts: new Date().toISOString(),
        service: "tax-service",
        level: "error",
        event: "schema-violation",
        context: result
      });
    }
    return result;
  }
  // Debug API
  async getDebugState() {
    return { cacheHits: this.cacheHits, offline: this.offline };
  }
  async setDebugFlag(flag, value) {
    if (flag === "offline") this.offline = value;
  }
}

Portable Telemetry Helper

// telemetry.js
export function emitTelemetry(event) {
  const payload = JSON.stringify(event);
  if (typeof window !== "undefined") {
    navigator.sendBeacon?.("/telemetry", payload);
    console.log("Telemetry (browser):", payload);
  } else {
    process.stdout.write(payload + "\n");
  }
}

The Cross-Runtime Test Pack

// tests/taxServiceTests.js
import assert from "assert";
import schema from "../taxService.schema.json" assert { type: "json" };

export const tests = [
  {
    name: "Basic Tax Calculation",
    run: async (runtime) => {
      const svc = await runtime.load("tax-service");
      const result = await svc.calculateTax({ amount: 100, region: "CA" });
      assert(result.total > 100, "Total should include tax");
      assert(schema.output.validate(result), "Output must match schema");
    }
  },
  {
    name: "Offline Mode",
    run: async (runtime) => {
      const svc = await runtime.load("tax-service");
      await svc.setDebugFlag("offline", true);
      const result = await svc.calculateTax({ amount: 50, region: "CA" });
      assert(result.source === "cache", "Expected cache source in offline mode");
    }
  }
];

The Cross-Runtime Runner

// runTests.js
import { runtimes } from "./availableRuntimes.js";
import { tests } from "./tests/taxServiceTests.js";

for (const runtimeName of runtimes) {
  console.log(`Running tests on ${runtimeName}...`);
  const runtime = await import(`./runtimes/${runtimeName}.js`);
  for (const test of tests) {
    try {
      await test.run(runtime);
      console.log(`✅ ${test.name}`);
    } catch (err) {
      console.error(`❌ ${test.name}: ${err.message}`);
    }
  }
}

Simulating Two Runtimes

availableRuntimes.js might just look like this for now:

export const runtimes = ["browser", "cloud"];

And each runtime adapter just needs a load function that returns a new instance of the service.

// runtimes/browser.js
import { TaxService } from "../taxService.js";
export async function load() { return new TaxService(); }

// runtimes/cloud.js
import { TaxService } from "../taxService.js";
export async function load() { return new TaxService();

Running the Loop

Run tests across both runtimes:

node runTests.js

1. Inspect telemetry output (in browser console or in CLI logs).
2. Manually set debug flags and re-run tests to simulate conditions.
3. Add failing production cases to the test pack and re-run

This example demonstrates that with just a bit of structure, you can run the same service across multiple runtimes, test it, debug it, and observe it without any environment-specific rewrites.

Takeaways

Portable services require portable tooling. If a service can run in a browser, on mobile, at the edge, and in the cloud, then your tests, debugging hooks, and telemetry must follow it without rewrites or custom adapters. UMA makes this possible by treating these capabilities as part of the service contract rather than optional extras.

In this post, we covered:

• Why traditional debugging and testing break down in a cross-runtime world.
• How UMA’s contract-first approach makes testing and debugging portable.
• How observability becomes a first-class feature with a unified telemetry schema.
• How to connect testing, debugging, and telemetry into a self-reinforcing feedback loop.
• A hands-on example showing the entire loop in two different runtimes.

The main shift is to consider runtime consistency not just for service execution but for the entire lifecycle, from local testing to production monitoring. Once your tooling is as portable as your services, you gain the confidence to move quickly without losing track of what is happening across your architecture.

If you are building distributed, portable systems today, start by making one of your core services testable, debuggable, and observable in more than one runtime. Once you have that loop working, scaling it to your entire stack becomes much easier.

Distributed systems are harder to trust when each runtime is its own black box. The fastest way to regain that trust is to make testing, debugging, and observability portable, just like the services themselves.

Here’s a 30-day challenge to put these ideas into practice:

1. Pick one core service in your stack that runs in more than one environment.
2. Write a simple contract (JSON Schema or equivalent) for its inputs and outputs.
3. Run a basic test suite for that service in two different runtimes without changing the tests.
4. Add a telemetry emitter that works in both runtimes and includes trust metadata.
5. Feed one anomaly from telemetry back into your tests so your loop closes automatically

Do this for one service, and you’ll see the pattern emerge: contracts remove environment guesswork, telemetry keeps production visible, and the feedback loop keeps your system improving without slowing down.

If you can prove this loop works in one place, scaling it across your architecture becomes a matter of repetition, not reinvention.

Coming Next In this post, we examined how UMA integrates testing, debugging, and observability into the same portable model as the services themselves, closing the gap between development and production. But understanding what happened is only half the story.

What happens when you need to react in real time? How do you build systems that respond automatically to anomalies, enforce policies without human intervention, and adapt their behavior safely across different runtimes?

The next post in this series will explore policy-driven automation. It will demonstrate how UMA can embed runtime policies that respond to telemetry, enforce guardrails, and even trigger self-healing workflows, all without relying on a centralized control plane. We will provide concrete examples of automated responses, policy validation, and rollback strategies that ensure distributed, portable services remain both safe and fast.

[embed]Resilience at the Edge How UMA brings guardrails, policies, and self-healing to every devicemedium.com

Found this valuable?

If this post changed how you think about runtime observability or distributed systems debugging, hit the clap button. It takes one second, and it helps other engineers find the series. 👏

📚 The Book

Everything in this post goes deeper into **Universal Microservices Architecture**: the runtime model, the full LifecycleRecord implementation in Rust and TypeScript, all six companion labs, and the complete five-part architecture from contract design to production deployment.

Universal Microservices Architecture | Get the book

Following along?

This post is part of The Rise of Device-Independent Architecture: a weekly series on building systems that run anywhere, scale deliberately, and don’t break when the environment changes.

[embed]The Rise of Device-Independent Architecture Explore how to build portable microservices that run across browsers, devices, and the edge. This series introduces…medium.com