Photo by Dewang Gupta on Unsplash

Beyond the Vector DB: Grounded, Auditable Agents with Amazon Bedrock AgentCore and OKF

The default architecture for enterprise RAG is predictable: chunk the text, embed it, sync the vectors to a managed store, and do a nearest-neighbor lookup at query time. That works for fuzzy search over documentation. It falls apart on structured, high-stakes data where the answer has to be exactly right and fully traceable financial controls, CVE exposure, clinical registries, and government survey statistics. Semantic distance cannot prove a control passed, cannot respect a skip-pattern, and will confidently hallucinate a number.

What if grounding were enforced by what physically ships in the deployment, not by prompt engineering? By pairing OKF (Open Knowledge Format) with Amazon Bedrock AgentCore, you ship the verified knowledge as plain files inside the deployable artifact no vector database, no embeddings, no chunking service and the agent literally cannot answer with anything that wasn’t verified.

8+ network transits per query across 5 isolated services

2+ transits per query retrieval runs in-process, zero sync

Full source: github.com/andrewwint/nhisokfchat. Every code block below is the real code, and the agent responses are captured live from the deployed runtime.

Inside the Sandbox: Local TF-IDF and Local Engine

How does an agent query files and databases locally without spinning up a heavy, slow infrastructure footprint? We split the labor between two ultra-lean, in-process tools running directly inside the container runtime:

1. Local TF-IDF

The container runs an in-memory TF-IDF search engine (using scikit-learn's TfidfVectorizer). Because the verified OKF corpus is small and curated (dozens or hundreds of files, not millions of raw documents), document indexing and cosine similarity computations are entirely deterministic, use minimal memory, and execute in under 1ms directly in-process.

2. Local Engine

For structured data, the Local Engine uses an analytical file-based library (like PyArrow, Pandas, or DuckDB) to run highly optimized queries against a compressed binary schema (such as a local .parquet or SQLite file) bundled beside the code. For example, a raw, bulky 29.4 MB survey dataset can easily be converted to Parquet and stripped of unused columns, shrinking it down to an incredibly lean 314 KB local slice.

This in-process architecture is perfect for starting small and adhering to strict data engineering principles. If your data scale eventually outgrows a local binary slice, this setup cleanly upgrades to storing larger Parquet files in Amazon S3 or querying a managed database, using the Local Engine as a highly secured, parameterized gateway that executes remote API, SQL, or MCP queries for real-time data while returning only safe, aggregated metrics back to the LLM.

The Idea: Verified Knowledge as Files

To demystify how these files appear: they are the output of a standard, pre-deployment data pipeline that extracts raw data from your sources (microdata files, databases, or API feeds), runs the necessary statistical calculations or assertion logic, and transforms them into the desired format.

An OKF bundle is simply a flat directory of these pipeline-generated Markdown files with YAML Frontmatter — one file per verified concept. A file is written to the directory only if its claim was programmatically checked by the data pipeline at build time against the source of truth. If the check fails, the concept is quarantined; the pipeline refuses to output the file.

Because it doesn’t exist as a file in the artifact, no tool can fetch it. Grounding is a property of the artifact, not a promise in the prompt. A concept is just a verified fact with its provenance and its check attached:

---
id: DIBINS_A
title: "Currently takes insulin (among adults with diagnosed diabetes)"
analytical_universe: "DIBEV_A == 1"  # who the figure is about
source: "NHIS 2023 Sample Adult public-use file (adult23.csv)"
value_pct: 31.96
verification:
  verdict: PASS  # this number was RUN and checked at compile time
  method: execution-grounded
  correct_pct: 31.96
---
Currently takes insulin (among adults with diagnosed diabetes), 
survey-weighted (WTFA_A).

That’s the whole trick: move verification upstream to compile time via the data pipeline, and let the deployment carry only what passed.

This Is a Pattern, Not a Health Data Demo

The example here compiles CDC survey data, but the shape is domain-agnostic: verified concept files + a thin agent whose two tools are grounded-or-refuse. Only the tool bodies change per domain.

• Audit & Accounting: Each control is an OKF markdown+YAML file. It acts as the immutable schema, defining the control criteria (e.g., “No transaction exceeds $10,000 without dual-authorization”) and containing the pre-approved query signature. The ledger lives in SQL. The agent is given two tools: search the verified controls, and run a parameterized, allow-listed query against the ledger (e.g., checking for un-authorized transactions). Asking “Is control 4.2 satisfied this quarter?” is resolved safely because the agent uses the static file to understand the criteria, and the live tool to execute the pre-approved query signature against the active SQL database. The agent can never write arbitrary SQL or hallucinate compliance parameters.
• Security: Each CVE exposure is an OKF document. It serves as the vulnerability database, holding the verified advisory, the affected package names, and the vulnerable version ranges (e.g., dependency-xyz < v2.4.1). The tools are MCP or API calls to active Git servers and static-analysis scanners to check the current package-lock.json in the production branch. Asking "Are we exposed to CVE-2026-1234?" matches the active dependency footprint against the verified vulnerable version ranges, guaranteeing no false alarms on clean configurations.
• Public-Health Statistics (This Demo): Concepts are verified survey figures; the second tool computes a survey-weighted subgroup from a slim data file that ships beside the code.

In every case, the skeleton is identical two tools, one that retrieves a verified concept and one that queries live data, wired to a grounded-or-refuse agent.

One rule survives every domain: because the agent composes the live query from user text, that query must pass an allow-list before it reaches any evaluator SQL, a DataFrame, a shell, or a git API. Hand raw model output to an evaluator and you have built an injection sink; validate it against a known grammar first and you haven’t.

The Agent, Assembled: main.py

Before we jump directly into the codebase, let’s visualize how these components communicate in-process. The entire agent runtime operates as a highly secured, local sandbox:

This visual outlines the entire transferable skeleton. Swap the two tool bodies for your domain (controls + SQL, CVEs + git) and everything else is unchanged. Note the real AgentCore + Strands API: BedrockAgentCoreApp, @app.entrypoint, and the Strands @tool + Agent.

# app/nhisokfchat/main.py
from bedrock_agentcore.runtime import BedrockAgentCoreApp
from bedrock_agentcore.runtime.models import PingStatus
from strands import Agent, tool
from strands.models.bedrock import BedrockModel

from nhis_okf import chat, config, helpers

app = BedrockAgentCoreApp()

# --- The two tools the agent can call. Thin wrappers over your domain logic. ---
@tool
def tool_search_okf(query: str) -> str:
    """Retrieve a matching verified concept from the OKF bundle 
    (aggregate figures only). Here that's in-process TF-IDF over 
    the markdown bundle — no vector DB, no embeddings; swap this 
    body for your domain's retrieval."""
    return chat.search_okf(query)

@tool
def tool_analyze_rows(variable, universe, stat="prevalence", q=0.5) -> str:
    """Run the live, allow-listed query and return an AGGREGATE 
    answer — never raw records. Here that's a survey-weighted 
    pandas/pyarrow query over the parquet bundled in the CodeZip;
    swap this body for a SQL query or an API call in your domain."""
    return chat.analyze_rows(variable, universe, stat=stat, q=q)

# --- The Strands agent: this object IS the reasoning loop. ---
def build_grounded_agent() -> Agent:
    """Claude on Bedrock + the grounding rules + the two tools. 
    Claude reads the question, picks a tool, calls it, and answers 
    from ONLY what the tool returns."""
    return Agent(
        model=BedrockModel(model_id=config.bedrock_model_id(), 
                           region_name=config.aws_region()),
        # the rules: grounded-or-refuse
        system_prompt=chat.OKF_ANALYST_PROMPT, 

        # the only things it can call
        tools=[tool_search_okf, tool_analyze_rows], 
    )

@app.entrypoint
def invoke(payload: dict, context=None) -> dict:
    """Answer one question, grounded in the verified bundle."""
    question = (payload or {}).get("question") 
                        or (payload 
                        or {}).get("prompt")
    if not question:
        return {"error": "no question provided", "answered": False}
    hits = helpers.retrieve(question) # grounding + citations
    try:
        text = str(build_grounded_agent()(question)).strip()
        return {"answered": True, 
                "mode": "generative", 
                "answer": text,
                "citations": [helpers._citation(h) for h in hits]}
    except Exception: # degrade to a cited, bundle-only answer
        ans = helpers.extractive_answer(question, hits)
        return {"answered": True, 
                "mode": ans.mode, 
                "answer": ans.text, 
                "citations": ans.citations}

@app.ping
def ping() -> PingStatus:
    return PingStatus.HEALTHY

That is the entire pattern. The two @tool functions delegate to domain logic (chat.search_okf reads the verified bundle; chat.analyze_rows runs the allow-listed live query and refuses an unverified field). Both return answers, never raw records, and if Bedrock is unavailable, the agent still returns a cited answer straight from the bundle. To retarget this to audit or security, you rewrite those two functions and the concept files — not the agent.

Deploying to Amazon Bedrock AgentCore

AgentCore Runtime accepts a direct-code (CodeZip) deployment up to 250 MB compressed enough to carry the verified bundle and the data (here a 314 KB file) inside the artifact itself. Your knowledge base is the deployable; there is no vector store to provision, sync, or pay for.

# build the CodeZip locally → 145.72 MB (< 250 MB limit)
agentcore package     

# → CDK → CloudFormation → AgentCore runtime
agentcore deploy -y  

# converse with agent grounded on OKF data and parquet files with hard data
agentcore invoke \
  '{"question": "What share of U.S. adults with diagnosed diabetes take insulin?"}'

# tear it down
aws cloudformation delete-stack --stack-name AgentCore-nhisokfchat-default   

Real, live output — actual agentcore invoke responses from the deployed runtime (mode: generative), then torn down:

1. Grounded from OKF files

Source: https://github.com/andrewwint/nhisokfchat/blob/main/docs/SAMPLE.md

2. Grounded from Parquet

Source: https://github.com/andrewwint/nhisokfchat/blob/main/docs/SAMPLE.md

2. Refusal

Source: https://github.com/andrewwint/nhisokfchat/blob/main/docs/SAMPLE.md

Grounding makes the agent less willing to guess than a strong ungrounded model — and every figure it serves passed verification at build time.

Pragmatic Data Engineering: Start Small, Scale Cleanly

To fit comfortably within AgentCore’s 250 MB CodeZip threshold, we didn’t just dump raw files into the zip. Instead, we followed core data engineering practices: start small, optimize local formats, and scale to enterprise infrastructure only when the physical limits demand it.

The strategy you choose should scale with your data footprint:

• Large Datasets: Storing files in Amazon S3 or querying a managed database is the correct, forward-thinking architectural progression.
• Small to Intermediate Datasets: Highly optimized formats like Parquet, SQLite, or compressed binary schemas are perfect. For very small configurations, simple CSV files work as well.

In our demo, we converted a raw, public CDC survey dataset (adult23.csv) weighing in at 29.4 MB to binary Parquet (adult23.parquet), instantly cutting its size down to 4.7 MB.

To go even lighter, we sliced away all extraneous survey columns we had no intention of querying, leaving only our target verified variables and weights. This final upstream transformation shrank our microdata footprint to a mere 314 KB(adult23_slice.parquet) representing a 99% size reduction from the source CSV.

By engineering our data upstream, our active query engine, code dependencies, OKF summaries, and live data slices package cleanly inside a tight, ultra-fast local sandbox.

The Size Constraints: The Runtime is Not the Code Interpreter

The most useful thing we learned came from a failed optimization. The AgentCore Code Interpreter ships a rich data-science stack pre-installedPandas, NumPy, scikit-learn, PyArrow, and others. Our CodeZip bundles the same stack (most of its 145.72 MB), so the obvious move was to delete those from pyproject.toml since they're "already installed."

ModuleNotFoundError: No module named 'sklearn'

The environment split visualized in the figure above explains why: the pre-installed libraries live exclusively inside the Code Interpreter a separate, network-isolated sandbox the agent sends code to. The Runtime (where your main.py boots) starts entirely blank and installs only what you declare; strip its dependencies, and your imports die at startup.

The path to a lean runtime isn’t stripping the artifact; it’s shifting the architectural boundary. Keep the bootstrap runtime tiny, stage data in S3, and dispatch heavy Pandas/pyarrow operations to the pre-provisioned Code Interpreter sandbox. For a getting-started agent, however, keeping everything in-process inside a 145.72 MB build is simpler, faster, and deploys comfortably under the 250 MB container ceiling.

The Architectural Win

Trading a heavy, multi-moving-part vector platform for an in-process, execution-grounded pattern on AgentCore changes the engineering equation. When your data and security constraints mandate absolute accuracy, this self-contained design buys you four distinct, production-grade advantages:

1. A Classic Architectural Trade-Off: Fielding’s Statelessness at Deploy Time

In his seminal dissertation, Roy Fielding defined REST’s stateless constraint to require that every request contain all information needed to process it, relying on no stored server context. While this trades network efficiency for larger request headers, it dramatically maximizes scalability and reliability.

We are applying Fielding’s core trade-off to deployment topology. By bundling all analytical libraries, data slices, and verified concepts directly inside our 146 MB CodeZip, we accept a larger deployment package to buy absolute scalability and execution reliability. Just as a stateless REST request can be processed independently by any server, our in-process agent execution is completely decoupled, requiring zero out-of-band databases, session caches, or cluster connections.

2. Eliminating the Network Tax on User-Perceived Performance

Fielding’s dissertation emphasizes that user-perceived performance is optimized when we minimize component interactions. In a traditional database-driven RAG setup, every user question triggers an expensive multi-hop journey: LLM to Embedding API, runtime to Vector Database, context construction, and back to the LLM.

By packaging our verified OKF bundle and data engine locally inside the CodeZip container, we completely bypass these remote network interactions. The lookup and aggregate compute layers run in-memory within the isolated runtime boundary. Communication latency is slashed to near-zero, enabling an incredibly snappy, responsive client experience.

3. Atomic, Version-Controlled Truth

Your knowledge base is your application package. Re-deploying your CodeZip updates your codebase, configurations, and verified statistics simultaneously, atomically, and side-by-side. There are no dangling database states, out-of-sync indexes, or drift between code and data.

Conclusion

We only scratched the surface here with two tools, four concepts, one dataset. But the main leverage is the upstream pipeline that produced the verified OKF files. Think SASS or LESS: you write a structured superset, and a build step compiles it into something clean and correct. The pipeline you build to create the OKF files runs the compile-time verifiers to check every claim, and focused, verified concept files come out. Correctness is enforced once, so the runtime never has to guess.

That changes what gets vectorized not arbitrary character-count slices of raw documents, but one pre-verified concept per file. And for questions with no precomputed concept, the second tool computes a deterministic, allow-listed aggregate from a bundled parquet today, or just as easily a SQL warehouse, data API, or MCP tool tomorrow. Verified concepts for what you anticipated, live aggregates for what you didn’t.

AgentCore supplied the runtime a containerized home for the agent, and a CodeZip that ships the knowledge beside the code, no vector cluster to operate. That’s a great delivery mechanism. But the delivery mechanism isn’t where you start. Start with the compile.

Source, deploy scripts, and the verified bundle: **github.com/andrewwint/nhisokfchat**. The compiler and execution-grounded verifier that produce the bundle live in the companion lab repo, **github.com/andrewwint/nhis-okf-compiler**.