Tinkering with Databricks: An Agent, MLflow, and a Hantavirus Registry

Continuing my tinkering, I turned my curiosity this time to the ecosystem around observability, evaluation, logging and tool use — the staples of a production-grade agentic workflow.

I would always get those recommendations in Databricks notebooks about MLflow and how cool it is and the fact that it sits within so seamlessly. And then I decided to actually test that claim.

So I built something real enough to stress-test it — or rather, feel connected to what I was testing. Not a toy notebook (though with coding assistants, that phrase has a changed frame of reference!). Actual API calls, tool use registered in Unity Catalog, a reasoning loop, and some curious questions on Hantavirus trials — enough to justify it not being a toy.

So what I decided in favor of was a clinical trials search agent. The ClinicalTrials.gov REST API is public, well-documented, and the data is genuinely interesting. The use case being real patients, researchers, and coordinators who spend hours navigating that registry. And the question “what trials are currently recruiting for this condition?” is the kind of thing an agent should be able to answer well — with citations, not just vibe.

The Build

I chose Databricks on Azure (as always, past predilections and comfort). The idea was simple and basic: seed a Delta table with ~100 trials across a few conditions, register two Unity Catalog functions (one SQL query against the cache, one Python function calling the live API), wrap them as LangChain tools, and point a reasoning model at them.

The part that genuinely impressed me first was UCFunctionToolkit. One line. You give it your Unity Catalog function names and it hands them to LangChain as tools — with the Unity Catalog governance and access-control model behind them. I expected more ceremony. There was not any.

Functions - the to-be tools! Stored in UC.

Just “tooled”! No ceremony!

Then MLflow Happened!

This is the part I actually set out to test, and glad I did as it did not disappoint.

I called mlflow.langchain.autolog() before running the agent. That’s it. Every subsequent invocation produced a full trace — which LLM call, which tool, what input, what output, how long each step took. Visible in the MLflow UI immediately.

The direct support for MLflow makes observability anything but an afterthought!

Then I ran evaluation. mlflow.evaluate(model_type=”databricks-agent”) spins up what Databricks calls Agent-as-a-Judge, a second LLM scores each answer on relevance, safety, etc. No labelled dataset required. (Though I came to know we can create our custom metrics too (like keyword-based ones). You give it questions; it runs them through your registered model and scores the outputs.

I ran five questions. Two errored — the free tier model endpoint hit its rate limit during concurrent eval calls, which is a real constraint worth knowing about. The three that completed gave me enough signal for a first-pass evaluation. Groundedness especially — the agent cited real NCT IDs, not invented ones. That’s the thing one actually cares about in clinical research.

The eval results, the traces, the model version — all linked. All reproducible. That part just worked, and I wasn’t expecting it to feel that clean.

Five questions for agent as a judge evaluation

Evaluation Results!

Where Genie Helped (Genuinely)

I used Databricks Genie throughout. Mainly to unstick myself when things went sideways.

I can imagine combining it with Claude code or Codex and realize the synergy!

One thing I hadn’t tried before (feels like, missed it): after Genie fixes a cell, type /doc in the chat. It reads the cell and adds inline comments explaining what each section does. Very useful in general, more so these days!

Just type /doc and see the comments appear. Comments in code have never been so painless and …necessary!

What Could Be Better

Honest now. The dependency situation seemed to me a bit rough. In my environment, databricks-langchain 0.19.0 breaks with langgraph 1.0.x. The error message points nowhere useful. Genie also did not point this out that seamlessly. The fix — pip install “langgraph<1.0” — is something you arrive at through trial and error. A published compatibility matrix (with Genie having access to in real time) would save people time.

Also, the tutorials still show AgentExecutor. In my setup, AgentExecutor did not work cleanly with mlflow.langchain.log_model, and I had better results moving toward LangGraph and model-from-code logging.

And if you’re on a trial workspace: agents.deploy() is not available. You discover this at the very last step. Finding out at the end is tad frustrating.

The Hantavirus Angle

One of my five eval questions was about Hantavirus trials. I included it because the virus class has been in the news.

The agent found NCT04323904 — the Hantavirus Registry (HantaReg), run out of the University of Cologne. Currently recruiting, not a drug trial but a registry collecting standardized clinical and epidemiological data on hantavirus infections, including disease presentation, diagnostics, treatment pathways, and outcomes. For such outbreaks this kind of registry would be genuinely useful.

Model responding to Hantavirus trial question from ClinicalTrials.gov

That’s the thing about building on real data. Gives some stories. Anyways my registered model looks like this in UC:

Alias like those you see make calling them convenient!

So, to conclude…

Though I really like to serve the model and test it via REST API. But for now, it was worthwhile exploration. MLflow earned its reputation. Loved it. Genie is more useful than I expected. The rough edges are real but fixable.