Reframing clinical data transformation: The role of agentic AI

By Sowmyanarayan Srinivasan, Sagar Madgi and Ansh Srivastava

Industry evolution: From human programming to agentic workflows

Clinical programming is progressing along an AI-autonomy evolution from AI-assisted execution to agent-driven workflows with human validation, and eventually to AI-driven delivery with humans providing governance, assurance and strategic oversight. (See Figure 1). Early AI implementations focused on discrete task assistance: metadata extraction, code snippet generation or rule-based checks. These approaches showed promise in pilots and targeted cases. However, scaling them reliably often required significant development effort relative to the value delivered.

Organizations need more than better models to address these challenges. They need systems that combine transparency, adaptability and human oversight. Agentic architectures deliver this by coordinating specialized agents across the workflow while making decisions and transformation logic visible. The result is a scalable and governable approach to end-to-end clinical programming, reflecting a broader shift across life sciences organizations. As ZS outlines in its perspective on agentic AI — *Agentic AI in pharma: Designing decision systems for an intelligent enterprise*, the next wave of transformation is not about deploying more AI tools, but about redesigning how recurring, high-impact decisions are made and executed across value streams. Clinical data transformation is one such decision system where agentic architectures enable continuous sense, decide and act loops, governed by human oversight and designed to scale with trust rather than fragment into isolated automation.

Figure 1

Structural complexity in clinical data workflows

Clinical trial statistical programming is a critical-path activity in phase III drug development and has historically been highly labor-intensive. A single study requires eight to 15 ADaM domains and 100–500 TLFs for regulatory submission, demanding 12 to 24 FTE-months of programming effort with 30%–50% quality control overhead.

This QC burden is further quantified as independent double programming consuming 1.6–2.0 times the primary programming effort, with iterative refinement cycles extending development timelines by 20%–40%. The process remains largely manual, dependent on scarce and expensive SAS programming talent, and burdened by rework arising from specification changes and quality concerns.

The deeper challenge is structural. SDTM-mapping decisions flow into ADaM derivations, which shape downstream TLF outputs, creating tightly coupled dependencies across the pipeline with limited traceability between stages. Automation approaches and standalone AI tools help address isolated tasks such as metadata extraction, code generation or validation checks, but struggle to operate reliably across interconnected workflows. Understanding why clinical programming has resisted large-scale automation requires looking beyond individual tasks and examining the workflow structure itself.

Traditional automation and standalone AI perform well when tasks are discrete and rules are explicit. Clinical data workflows require more than that and here is why:

Standards that aren’t truly standard CDISC provides a framework, not a recipe. The same patient data can map to different domains across studies depending on therapeutic area, sponsor conventions and protocol design. These decisions require contextual interpretation, limiting the effectiveness of fixed rules and reusable templates.

Specifications written in natural language Mapping logic, population definitions and analysis rules are documented in Word files, PDFs and spreadsheets. Translating statements such as “patients who received at least one dose and had a postbaseline assessment” into executable logic requires systems that can interpret ambiguous clinical language within broader study context.

The work is a chain, not a series of independent tasks Each stage depends on upstream context including protocol criteria, sponsor conventions and cross-dataset consistency. Decisions made during SDTM-mapping influence ADaM derivations and downstream TLF outputs, making isolated task automation insufficient for maintaining consistency across the workflow.

Validation failures require iterative investigation

When datasets fail validation, resolution requires tracing earlier mapping decisions, understanding their rationale and refining downstream outputs while preserving quality and consistency across the workflow. This process depends on iterative reasoning, coordinated updates and targeted human review rather than one-time code generation alone.

Changes midstudy are expensive Protocol amendments and specification updates propagate across mappings, derivations, populations and analysis outputs. Assessing impact requires awareness of workflow dependencies and prior transformation decisions across multiple stages of execution.

Together, these characteristics create an environment where traditional automation and standalone AI often fall short. The challenge is not task complexity. Instead, dependencies are implicit, specifications are ambiguous and much of the context resides in human judgment.

Approach: A reusable agentic framework

The problem has not changed. Our ability to solve it has. Traditional automation struggled with the context, variability and ambiguity present in clinical data workflows. Large language models introduce a fundamentally different capability. They can read Statistical Analysis Plans written in natural language, reason about embedded clinical and conditional logic, generate transformation code, evaluate outputs, and iterate when discrepancies arise. This is not because each scenario is explicitly programmed, but because the system can generalize across patterns in clinical logic and code. This shift makes it possible to move beyond task-level automation toward workflow-level orchestration.

Three principles guide our approach.

Modularity as a foundation for resilient systems

The pipeline is built from discrete, independently functional components — including specialized agents, transformation libraries, validation routines — each with a defined scope and a defined interface. When something changes during an engagement, whether a CRF structure diverges from the specification, a protocol amendment arrives or a sponsor convention turns out to be more idiosyncratic than expected, the impact remains contained. The affected component can be addressed without disturbing the rest of the pipeline. Real-world clinical studies rarely conform perfectly to design assumptions. A modular system handles these variations gracefully. A monolithic system does not.

Study-agnostic reusability, configuring instead of rebuilding

The agents are designed to work across studies while adapting to sponsor conventions, protocol requirements and therapeutic-area differences through configuration rather than rebuilding. This enables teams to reuse a common core system while still supporting study-specific requirements. This approach also creates a strong foundation for evaluating whether outputs are driven by scientific rationale rather than learned behavioral patterns, improving transparency, consistency and governance across studies.

Human oversight as a design principle, not an add-on

Every agent has defined escalation points at which it pauses, surfaces its reasoning and waits for human input before continuing. Confidence thresholds determine when the system operates independently and when decisions are routed to a reviewer. Reviewer interfaces show not just the output but also the underlying rationale, intermediate steps, flags raised and alternatives considered. As a result, human judgment is applied where it creates the greatest value, and at the point where intervention is most effective.

Figure 2 presents a reference architecture that brings a multiagent approach to clinical programming to life by coordinating specialized agents across the end-to-end workflow.

Figure 2

The following are key architectural highlights:

Separation of orchestration from domain execution

The architecture separates AI-driven workflow orchestration from statistical execution. AI agents manage coordination, reasoning and workflow logic, while SAS and R perform domain-specific transformations and analysis generation. This allows workflows to remain flexible while keeping outputs interpretable, reviewable and aligned with existing clinical validation and regulatory processes.

Stateful multiagent coordination with human oversight

The platform is designed around stateful agent orchestration rather than isolated task execution. Specialized agents coordinate across SDTM, ADaM and TLF workflows while retaining context, tracking dependencies and routing decisions through human-in-the-loop review points where interpretation or validation is required.

Model-agnostic and extensible execution framework

LLMs are accessed through a unified gateway supporting multiple providers and interchangeable execution components. Shared services such as MCP context management, agent memory, service registries and reusable tool integrations allow the architecture to evolve alongside rapidly changing AI ecosystems without requiring major redesign.

Enterprise-grade governance and operational foundations

Observability, authentication, security controls, configuration management and scalable infrastructure are embedded as foundational capabilities rather than added later. This ensures traceability, operational resilience and production readiness for regulated clinical data transformation workflows.

Key considerations for creating value

The path to value creation is shaped as much by implementation choices and organizational readiness as by advances in AI itself. The following considerations are particularly important.

The constraint is the specification, not the model Output quality is driven more by specification curation than model capability. A stronger model working on a poorly curated specification consistently underperforms a weaker model working on a well-structured one.

Test against real-world complexity from the start Synthetic data proves that a pipeline can execute, but not that it can handle real conditions. When applied to actual study data, issues emerge such as edge cases in derivations, cyclic dependencies and merge failures that are invisible in clean test environments. Production readiness is determined by performance under these conditions, not ideal ones.

Change management is the long game Technology evolves faster than organizations. Transitioning to agentic workflows requires new governance models, review protocols and definitions of quality. Organizations that scale successfully invest in both system deployment and organizational readiness in parallel rather than sequentially.

Common components are the real infrastructure The most durable gains come from shared foundations such as knowledge bases, retrieval systems, debugging frameworks and provenance tracking. These improve with reuse and should be treated as long-term infrastructure rather than engagement-specific assets.

A functional system in production creates more value than a perfect system that never launches. Early deployment of functional systems generates real-world evidence, builds confidence and accelerates learning. In practice, this enables an active learning loop, where system outputs, human feedback and real-world edge cases continuously refine both models and workflows.

Discovery before build is not optional A structured discovery phase ensures that automation targets the right problems at the right level of the pipeline. It clarifies where efficiency gains justify complexity and identifies foundational components that must be designed upfront rather than retrofitted.

What comes next?

Three developments will shape the next phase, and all three are already underway.

· Knowledge bases will evolve from engagement-specific tools into cross-sponsor industry infrastructure. Shared standards store validated across thousands of studies will replace assets developed for individual engagements. Emerging initiatives such as TransCelerate’s USDM and broader metadata standardization efforts are laying the foundation for interoperable clinical data infrastructure, where common models, reusable derivation logic and machine-readable study metadata can be shared, governed and reused across sponsors.

· Regulatory frameworks will mature to formally support agentic workflows. Organizations operating under rigorous internal governance today will be better positioned to engage with that process than those encountering these questions for the first time.

· Human involvement will remain critical, but its role will increasingly shift toward governance, oversight and exception handling as accumulated evidence validates agent outputs across engagements, therapeutic areas and derivation types.

Each of these developments rewards organizations that moved early and penalizes those that waited for certainty that was never coming.

A structured progression from a quick discovery to pilot to production deployment is critical for key business outcomes. The minimum viable step is not a technology evaluation or a proof-of-concept built on synthetic data, but a production-oriented pilot grounded in real study data, real governance requirements and human reviewers working alongside agent outputs. Such a pilot should have a clearly defined path to broader deployment. Its primary objective is to help the organization understand what is required to operate agentic workflows within its unique environments: data structures, sponsor conventions, governance requirements and review processes.

Agentic AI is primed to reshape clinical data transformation. The direction is becoming increasingly clear. The open question is whether organizations shape that transition or respond to it.

Read more insights from ZS.

This article reflects our personal views. They do not necessarily represent any official position of ZS.