Building an Agentic Orchestrator: From Strategic Goals to Automated Execution

How we built a system that decomposes high-level business goals into task DAGs executed by specialized AI subagents; with persistent memory, deterministic simulation, and a replay debugger.

The Problem: Strategy-to-Execution Gap

Every campaign team faces the same challenge: translating a strategic goal like “increase Q2 email engagement by 20% for our enterprise segment” into a concrete, sequenced set of actions. Today, this translation happens in meetings, spreadsheets, and Slack threads. Tasks get lost, dependencies are ignored, and when something fails, nobody can trace back to understand why.

We set out to build a system that closes this gap; an Agentic Orchestrator that accepts a high-level goal, decomposes it into a directed acyclic graph (DAG) of tasks, executes those tasks through specialized AI agents, and maintains a complete audit trail of every decision.

This post covers the architecture, the subagent design, the learned policy experiment, and the engineering practices that make it production ready.

Agentic Orchestrator

Architecture at a Glance

The system has four layers:

┌─────────────────────────────────────────────────────┐
│                   REST API (FastAPI)                │
├─────────────────────────────────────────────────────┤
│              Orchestrator Engine                    │
│     Planner → Retriever → Executor → Verifier       │
├─────────────────────────────────────────────────────┤
│              Task Graph Engine (DAG)                │
├─────────────────────────────────────────────────────┤
│  PostgreSQL    │    Redis     │    ChromaDB         │
│  (Audit/State) │ (Queue/Cache)│  (Vector Memory)    │
└─────────────────────────────────────────────────────┘

A client submits a goal via the API. The Orchestrator coordinates four subagents:

1. Planner — Decomposes the goal into a DAG of tasks
2. Retriever — Fetches relevant context from vector memory
3. Executor — Runs each task with retry logic
4. Verifier — Validates results against original constraints

The entire execution is logged to an immutable audit trail in PostgreSQL, and the DAG state is checkpointed to Redis for crash recovery.

Subagent Design: LangChain-Style Interfaces

Each subagent follows a common interface pattern inspired by LangChain:

class BaseAgent(abc.ABC):
    def __init__(self, name: str) -> None:
        self.name = name
        self.logger = structlog.get_logger(agent=name)

    @abc.abstractmethod
    async def run(self, input_data: dict) -> dict:
        ...

This makes agents composable and testable in isolation. Each agent receives structured input and returns structured output — no magic, no hidden state.

The Planner: Heuristic Rules Meet Reinforcement Learning

The Planner is the most interesting component. It works in two modes:

Heuristic mode matches the goal’s channels (email, in-app, SMS) to task templates:

TASK_TEMPLATES = {
    "email": [
        {"name": "segment_audience", "agent_type": "executor"},
        {"name": "design_email_template", "agent_type": "executor"},
        {"name": "personalize_content", "agent_type": "executor"},
        {"name": "schedule_send", "agent_type": "executor"},
        {"name": "monitor_delivery", "agent_type": "verifier"},
    ],
    # ... more channels
}

For a multi-channel goal, the Planner creates parallel chains; one per channel — with proper dependency ordering.

Learned mode uses a small REINFORCE policy (2-layer MLP, ~3,400 parameters) trained on the deterministic simulator. The state vector encodes the goal embedding, channel count, normalized budget, and execution progress. The action space selects which task template to add next.

This isn’t meant for production (yet); it’s a proof-of-concept showing how RL can improve planning decisions over time as the system accumulates execution data.

The Retriever: Vector Memory for Context

Before planning, the Retriever searches ChromaDB for relevant historical context:

async def retrieve(self, query: str, top_k: int = 5) -> list[dict]:
    results = self._memory.query(query_texts=[query], n_results=top_k)
    return [
        {"content": doc, "metadata": meta, "distance": dist}
        for doc, meta, dist in zip(...)
    ]

Over time, as campaigns complete, their results are stored back into vector memory. This creates a feedback loop: the system gets better at planning because it remembers what worked before.

The Executor: Retry-Aware Task Runner

Each task in the DAG is executed by the Executor agent. It supports:

• Custom handlers registered by task name prefix
• Default execution (simulated or real)
• Retry logic with configurable max attempts
• Timing for performance monitoring

The key design choice: execution happens in asyncio.to_thread(), so CPU-bound or I/O-bound tasks don't block the event loop:

await asyncio.gather(*[_run_single(t) for t in ready_tasks])

Independent branches of the DAG execute concurrently.

The Verifier: Trust, but Verify

After all tasks complete, the Verifier runs four checks:

1. Completion rate — Did >= 80% of tasks succeed?
2. No critical failures — Did monitoring/verification tasks pass?
3. Budget compliance — Did we stay within budget?
4. Channel coverage — Did we cover all requested channels?

Each check produces a confidence score. If the aggregate confidence drops below 0.7, the goal is marked as failed and can trigger re-planning.

The Task Graph Engine

The heart of the system is a DAG executor built on NetworkX:

class TaskGraph:
    def __init__(self):
        self._graph = nx.DiGraph()
        self._tasks = {}

    def get_ready_tasks(self) -> list[TaskNode]:
        """Return tasks whose dependencies are all completed."""
        return [
            task for task in self._tasks.values()
            if task.status == PENDING
            and all(self._tasks[dep].status == COMPLETED
                    for dep in self._graph.predecessors(task.id))
        ]

Key properties:

• Cycle detection on insertion (prevents DAG corruption)
• Topological execution respects dependencies
• Concurrent execution of independent branches
• Cascade skip — when a task fails, all downstream tasks are marked SKIPPED
• Serializable — the entire graph can be dumped to JSON for persistence or replay

Deterministic Simulator

One of the most valuable components for development is the deterministic simulator:

sim = Simulator(SimulatorConfig(seed=42, failure_rate=0.1))
sim.reset(goal, task_graph)
state = sim.run_to_completion()

Given the same seed and goal, it produces identical execution traces. This enables:

1. Reproducible testing — integration tests that never flake
2. RL training — generate thousands of episodes for policy optimization
3. Debugging — replay exact sequences in the UI
4. What-if analysis — “What happens if failure rate increases to 30%?”

The simulator models latency (normal distribution) and failure (Bernoulli) for each task, producing step-by-step traces.

Replay UI: Debugging Agent Decisions

We built a Streamlit-based Replay UI that lets operators:

• Visualize the task DAG with status coloring
• Step through execution one task at a time
• Inspect agent inputs, outputs, and confidence scores
• View the chronological audit trail
• Run simulations with different parameters

This transforms debugging from “grep through logs” to “click through the visual execution trace.”

Engineering Practices

Observability Stack

• Prometheus scrapes metrics from the /api/v1/metrics endpoint
• Grafana displays request rates, latencies, goal completion rates, and error rates
• Jaeger provides distributed tracing via OpenTelemetry
• Structured logging via structlog with machine-parseable JSON output

Testing Strategy

We maintain three test layers:

+--------------+-----------+----------------+-------------+
| Layer        | Count     | Dependencies   | CI Stage    |
+--------------+-----------+----------------+-------------+
| Unit         | 40+ tests | None           | Every push  |
| Integration  | 10+ tests | TestClient     | Every push  |
| E2E Smoke    | 3 tests   | Docker services| Main branch |
+--------------+-----------+----------------+-------------+

Unit tests cover the task graph, all four agents, the simulator, the policy network, and the audit logger. Coverage target: 80%.

Infrastructure as Code

The project ships with both Terraform (AWS ECS + RDS + ElastiCache) and Helm (Kubernetes) configurations, each with dev and prod variants. Cost-control flags keep dev deployments minimal (~$42/month on AWS).

Synthetic Data and Privacy

All data is synthetic, generated by data/generator.py with configurable anonymization:

python data/generator.py --goals 1000 --docs 500 --seed 42

No real user data is ever stored or processed. The generator includes privacy knobs for name anonymization and identifier hashing.

Performance

Load testing with Locust (50 concurrent users, 60 seconds):

+----------------+----------------+
| Metric         | Result         |
+----------------+----------------+
| Throughput     | 24 req/s       |
| Avg latency    | 38 ms          |
| p95 latency    | 120 ms         |
| Error rate     | 0%             |
+----------------+----------------+

The heaviest endpoint is /simulate (avg 150ms), which runs the full simulation synchronously. All other endpoints respond well under the 200ms SLO.

What’s Next

1. Production-grade policy training — Replace REINFORCE with PPO, train on real execution data
2. Streaming updates — WebSocket support for real-time goal status
3. Multi-tenant isolation — Namespace goals and memory per organization
4. External integrations — Connect the Executor to real marketing platforms (Braze, SendGrid, Iterable)
5. Human-in-the-loop — Approval gates for high-budget or high-risk tasks

Getting Started

git clone https://github.com/sourabh-virdi/agentic-orchestrator.git
cd agentic-orchestrator
cp .env.example .env
docker compose up -d
curl http://localhost:8000/api/v1/health

Submit your first goal:

curl -X POST http://localhost:8000/api/v1/goals \
  -H "Content-Type: application/json" \
  -d '{
    "title": "Q2 Email Campaign",
    "description": "Increase open rates by 20%",
    "constraints": {
      "budget_usd": 5000,
      "channels": ["email", "in-app"],
      "audience": "enterprise"
    }
  }'

Open the Replay UI at http://localhost:8501 to watch it execute.

About the Author: Sourabh Virdi has vast experience in the IT industry, specializing in AI/ML, Python development, and enterprise system architecture. My expertise spans Agentic AI Frameworks, .NET frameworks, identity provider (IdP) applications, and modern data warehousing solutions including Snowflake. Connect on LinkedIn.