Intelligent Data Steward: Leveraging AI Agents to automate data stewardship tasks

What if every employee had an on-demand data steward available through a simple chat interface?

Image from Piotr Kononow on https://dataedo.com/cartoon/data-owner-data-steward

Introduction

Nowadays, organizations collect data from an ever-growing range of sources, increasing both the volume of information available and the expectation that it can yield better and more informed decision-making. Making that possible requires effective data stewardship: maintaining data quality, documentation, discoverability, and access. In large organizations, these responsibilities often fall to data stewards, who operate at the intersection of technical infrastructure and business stakeholders.

The Stewardship Bottleneck

The challenge is that as data continues grow, so does the operational burden on these teams. In many large-scale organizations, data stewards are organized into groups of both technical and non-technical personnel, each fielding requests from analysts, engineers, and business users across the organization. The process regularly involves considerable manual effort and is complicated by communication delays, coordination overhead, and the constant challenge of routing the right question to the right person. The result is a function that is essential but difficult to scale and one where the gap between demand and capacity tends to widen over time.

If the stewardship bottleneck is driven by the growing volume of routine requests, the obvious question is: can those interactions be automated? The Intelligent Data Steward (IDS) is a multi-agent system designed to do exactly that.

Intelligent Data Steward: An AI Agent Based Architecture for Automated Data Stewardship

The Intelligent Data Steward is a multi-agent system that automates data stewardship tasks through a conversational interface. Instead of routing every data question through an overloaded team, IDS embeds stewardship capability directly into the user’s workflow. IDS acts as a permanent on-call expert for the data estate — one that understands natural language questions, knows where to look, and returns precise, validated answers without a ticket, a wait, or a handoff.

Architecture Overview

Intelligent Data Steward Architecture

The IDS is built on top of LangGraph and LangChain using a hierarchical multi-agent architecture. This design mirrors traditional data stewardship practices by breaking complex responsibilities into smaller, specialized functions. A key design decision was choosing a multi-agent architecture over a single agent equipped with all available tools. By assigning narrow responsibilities to specialized agents, IDS reduces cognitive load, improves reliability, and makes agent behavior easier to interpret and debug. This separation of concerns also allows new stewardship capabilities to be added without impacting existing components.

The frameworks LangGraph and LangChain were selected because they provide strong support for agent orchestration, structured tool usage, and stateful workflow management.

Top-Level Supervisor

The Top-Level Supervisor serves as the entry point of the IDS. Every user request passes through this agent first, making it responsible for understanding the user’s intent, selecting the most appropriate specialized team, and coordinating task execution.

Rather than performing domain-specific work itself, the supervisor acts as an orchestrator. It delegates tasks through structured handoff mechanisms, gathers the results produced by specialized teams, and transforms them into clear, user-friendly responses. This separation of responsibilities keeps the supervisor focused on coordination while allowing domain experts within the system to handle the actual processing.

A key design principle is that the Top-Level Supervisor remains unaware of the underlying sub-agents and their tools. Instead, it operates at a higher level of abstraction, routing requests based on intent rather than implementation details. This approach improves modularity and makes the architecture easier to scale and maintain.

The supervisor is implemented using LangGraph Supervisor, a library specifically designed for building hierarchical multi-agent systems. Its create_supervisor function provides built-in support for intent-based routing and task delegation, enabling seamless coordination between specialized agent teams.

Diagram of the Top Level Supervisor

Data Discovery Team

The Data Discovery Team is responsible for helping users understand and explore data assets before querying them. Managed by its own supervisor and invoked by the Top-Level Supervisor when discovery-related tasks arise, the team follows a hierarchical structure in which a dedicated supervisor coordinates a set of specialized worker agents.

Rather than performing discovery tasks directly, the Data Discovery Supervisor analyzes the request, determines which worker agent is best suited for the task, delegates work, collects structured outputs, and decides whether additional agents need to be invoked. This orchestration layer maintains a complete view of the discovery process while remaining unaware of the low-level tools used by individual agents.

The team consists of six single-responsibility agents, each focused on a different aspect of data understanding: schema extraction, semantic search, metadata validation, sensitive data detection, data quality assessment, and data exploration. For example, the Semantic Search Agent identifies datasets, tables, and columns that are conceptually related to a user’s request, while the Schema Extractor uncovers relationships between database entities.

To improve reliability, all agents return structured outputs validated through Pydantic schemas rather than free-form text. This design reduces hallucinations, enables automatic validation of results, and allows the supervisor to make informed decisions about the next steps in the discovery workflow.

Query Translation Team

The Query Translation Team is responsible for converting natural-language questions into validated and executable SQL queries. Whenever a user submits an analytical request, the team uses semantic search to identify the tables, columns, and business concepts most relevant to the user’s intent, bridging the gap between human language and the underlying database schema.

This team is composed of only one agent, the Query Translation Agent, which interprets the request, maps natural-language references to concrete database entities, generates the appropriate SQL query, and executes it against the target database. Before execution, the generated query is reviewed by a separate LLM-based validation step, which checks its syntactic correctness and helps identify potential issues. This additional verification layer improves reliability and reduces the likelihood of invalid queries reaching the database.

The agent combines LangChain’s SQL toolkit with domain-specific tools for schema exploration and value discovery, enabling it to generate accurate queries even when users are unfamiliar with the database structure. Once execution is complete, the results are returned to the Top-Level Supervisor in a structured format for response generation.

Diagram of the Query Translation Agent

User Interface

The Intelligent Data Steward features a web-based interface built with Streamlit, chosen for its simplicity, rapid development capabilities, and seamless integration with the underlying agent architecture.

Users authenticate through a registration and login system before accessing the platform. Once logged in, they interact with the IDS through a conversational chat interface, where they can ask questions, start new conversations, and review previous chat sessions through a sidebar navigation panel. Responses are generated in real time, with loading indicators providing feedback during processing.

To support continuous improvement, the interface includes a built-in feedback mechanism. Users can rate responses as helpful or not helpful, with negative feedback allowing them to provide additional comments. Rather than being stored permanently, this feedback is applied only within the current session, enabling the system to adapt its behavior while preserving the integrity of the global agent configuration.

The active conversational interface showing the exchanges between the user and IDS. It includes a focus on the binary feedback mechanism for response evaluation

From Question to Answer: An Example Workflow

Runtime execution of the IDS during a query translation task

To illustrate how the system operates, consider a business analyst asking:

“How many loan applications were submitted for each contract type?”

The request first reaches the Top-Level Supervisor, which determines that the question requires a data query. The task is then routed to the Query Translation Team.

Using semantic search, the Query Translation Agent identifies the tables, columns, and business concepts most relevant to the request. It then generates a SQL query, which is reviewed by a separate LLM-based validation step before execution. Once validated, the query is executed against the database and the results are returned in a structured format.

Finally, the Top-Level Supervisor synthesizes the results into a clear, natural-language response that can be understood by the user without requiring any knowledge of SQL or the underlying database schema.

Conclusion

The Intelligent Data Steward demonstrates how a hierarchical multi-agent architecture can automate many of the repetitive tasks traditionally performed by data stewards while maintaining transparency, reliability, and interpretability. By combining specialized AI agents, structured workflows, and a conversational user experience, IDS provides a practical blueprint for the future of intelligent data management.

This work was part of my Master’s thesis in “Data Science & Business Informatics” at the University of Pisa, conducted in collaboration with Data Reply.