From Local LangGraph to a Hosted AI Agent: What I Learned Deploying on DigitalOcean Gradient

Building the agent was straightforward. Making every layer agree on how it should run was where the real learning happened.

I started this project with a simple question:

Could an AI agent look at an application idea and recommend the right infrastructure path — not just for today, but for where that application might eventually scale?

The concept became the **DoggyDish Deployment Retriever**, a small LangGraph application that evaluates a workload and recommends one of three DigitalOcean deployment paths:

• Serverless Inference
• Dedicated Inference
• A GPU Droplet

This was not meant to become an all-knowing infrastructure architect. It was designed as a practical example of how an agentic application can begin with a few understandable decisions, generate a useful recommendation, and establish a path toward more powerful hardware.

The finished agent worked. It ran locally in Docker, deployed through DigitalOcean’s Agent Development Kit, called a hosted foundation model, and returned a structured recommendation through a public endpoint.

The journey there was more interesting than the final command suggests.

The successful deployment itself ultimately came down to:

$> gradient agent deploy

The DigitalOcean CLI validated the project, packaged the source, created a new staging release, monitored the deployment, and returned a hosted endpoint. Once the project was correctly structured, the actual deployment took only a few minutes.

But getting the application, container, environment variables, model client, and hosted runtime to agree with one another revealed several lessons about building agents on a platform that is still relatively new.

DigitalOcean’s Agent Development Kit, or ADK, is currently in public preview. DigitalOcean describes it as a Python SDK and CLI for taking agent code built with a developer’s preferred framework and deploying it as a hosted service. That combination — an emerging product with a remarkably simple deployment model — is what made this experiment compelling.

The application started with a decision, not a prompt

The most important architectural decision was to avoid asking the language model to choose the infrastructure on its own.

A model can write a convincing explanation, but that does not mean it should control a consequential routing decision.

Instead, the agent accepts four basic pieces of information:

• What the application does
• Whether its traffic is experimental, unpredictable, or consistently high
• Whether it requires a custom model
• Whether it requires operating-system-level control

LangGraph then applies a deterministic decision tree.

When the workload requires operating-system control, the agent selects a GPU Droplet.

When it requires a custom model or has steady, substantial demand, it selects Dedicated Inference.

Otherwise, it selects Serverless Inference.

Only after that route is selected does the language model enter the workflow. Its job is not to decide the architecture. Its job is to explain the already-selected architecture in a useful way.

That separation became the foundation of the application:

This is a small design choice with a much larger implication.

The graph owns the decision.

The model owns the communication.

That makes the result easier to test, easier to explain, and less likely to change unexpectedly because a model interpreted the same request differently.

Why these three infrastructure paths?

The three routes represent stages in the life of an AI application.

Serverless Inference

Serverless is the natural starting point when usage is irregular, the application is still being tested, or there is no need to manage the infrastructure behind the model endpoint.

DigitalOcean’s Serverless Inference service allows applications to send requests directly to available foundation models without first provisioning model-serving infrastructure. It is particularly appropriate for unpredictable or spiky demand.

Dedicated Inference

A workload can eventually become steady enough that dedicated GPU capacity makes more sense.

DigitalOcean positions Dedicated Inference for sustained, high-throughput workloads. It provides managed, dedicated GPU resources and lets the developer choose capacity and tune for characteristics such as latency, throughput, concurrency, and cost.

GPU Droplet

Some applications need more than a managed model endpoint.

They may require control of the operating system, drivers, container runtime, model-serving software, or a specialized stack such as vLLM. That is the point where a GPU Droplet becomes the more appropriate path.

The goal of the agent was not to declare that one route is universally better. It was to show that application requirements should determine how much infrastructure control the developer takes on.

The tools behind the project

The project used a relatively small collection of tools, each with a clearly defined role.

Python

Python contained the application logic, input parsing, model client, output validation, and DigitalOcean entrypoint.

LangGraph

LangGraph provided the stateful workflow.

The graph contained:

• A workload-classification node
• A Serverless recommendation node
• A Dedicated Inference recommendation node
• A GPU Droplet recommendation node
• Conditional edges that routed the state to the selected path

The graph was intentionally small. That made it easy to understand exactly why a request took a particular route.

Docker Desktop and Docker Compose

Docker created a consistent local environment.

Rather than relying on whichever Python packages happened to be installed on the Windows machine, the project pinned its dependencies and ran the application inside a Linux container.

Docker Compose became the repeatable wrapper around that environment. The same project structure could be built, tested, and used to run the Gradient CLI.

PowerShell

PowerShell was the operating console for the project.

It was used to:

• Build the container
• Invoke local endpoints
• Create JSON request bodies
• Call the hosted agent with Invoke-RestMethod
• Inspect environment values
• Confirm character lengths and encodings
• Run the Gradient CLI inside the container

DigitalOcean Gradient ADK

The ADK connected the local Python project to the hosted DigitalOcean environment.

The CLI handled project validation, dependency installation, entrypoint verification, source packaging, release creation, and deployment monitoring.

DigitalOcean’s documentation describes the ADK as framework-flexible: the developer can build with tools such as LangGraph and then use the ADK to host the resulting agent. The deployed application is exposed through an endpoint, while releases and runtime logs are available through the DigitalOcean interface.

DigitalOcean Serverless Inference

The hosted agent used DigitalOcean’s model API for generating the final recommendation.

The project used the AsyncGradient client and the openai-gpt-oss-120b model available through DigitalOcean’s inference platform. DigitalOcean maintains a model catalog for models usable across serverless inference, dedicated inference, agents, and ADK deployments.

Git

Git was added before finalizing the project so that source code and configuration could be versioned without committing the real .env file.

This became especially important because environment handling ended up being the hardest part of the deployment.

The first problem: the model sounded more confident than it was

The earliest version worked locally, but the recommendation exposed a common agent-development problem.

The language model filled gaps with architecture details that sounded plausible.

It introduced products, deployment mechanisms, service tiers, and numerical thresholds that were not part of the application’s approved guidance. Some of the language sounded authoritative enough to pass a casual review, but it was not grounded in the design.

This was a useful reminder:

A polished response is not necessarily a reliable response.

The fix was not simply to make the prompt longer.

The application was changed so the model received a route-specific guidance block that served as the exclusive source of infrastructure claims.

For example, the Serverless route told the model that:

• Docker Compose was for repeatable local development
• The DigitalOcean ADK hosted the LangGraph application
• No permanent Droplet was required
• The application would call Serverless Inference
• Dedicated Inference should be evaluated when measured demand became steady and substantial

The Dedicated and GPU Droplet routes received their own constrained guidance.

The system prompt also explicitly prohibited unsupported product claims and invented numerical thresholds.

That still was not enough by itself.

The output was passed through a validation layer that checked:

• The response was not empty
• The response used plain ASCII text
• Required headings appeared exactly once
• Headings appeared in the correct order
• Forbidden phrases did not appear
• Unsupported numerical claims were rejected

The final recommendation had to use four consistent sections:

1. Fetch This Architecture
2. Why It Fits
3. Build This First
4. Hardware Scale-Up Trigger

This changed the model from an open-ended architect into a constrained explanation layer.

The broader lesson was clear: reliable agents need more than prompts. They need application logic, bounded context, and programmatic validation.

Local success did not guarantee hosted success

Once the graph worked locally, I expected deployment to be the easy part.

In many ways, it was.

The ADK command validated the entrypoint, installed requirements, built the project, created a staging release, and returned an invocation endpoint.

But the first real request to the hosted agent returned:

500 Internal Server Error
failed to route request to agent

The deterministic routing logic worked.

The hosted runtime could start.

But the model call failed.

That meant the problem existed somewhere between the application container and the inference API.

The first diagnostic question was simple:

Could the hosted application actually see its environment variables?

Locally, the model key and model ID were present. The .env file was present inside the development container. Yet the hosted release reported that the model credentials were missing.

This led to the most important debugging discovery in the project.

The .env file existed—and still did not exist

The Gradient deployment package included .env.

Verbose deployment output confirmed that the file was being copied into the source archive.

But the Docker build had its own rules.

The project’s .dockerignore file excluded .env, which meant the deployment source could contain the file while the final application image did not.

Both observations were technically true:

• Gradient had packaged .env
• Docker had omitted .env

Removing .env from .dockerignore allowed it to be copied into /app in the hosted runtime.

This was a classic boundary problem. Every individual tool behaved correctly, but each tool had a different understanding of which files belonged in the final artifact.

The source package was not the same thing as the Docker build context.

The Docker build context was not automatically the same thing as the hosted filesystem.

That distinction is easy to miss when a deployment platform abstracts most of the infrastructure.

There was also an important security balance.

The real .env remained excluded from Git through .gitignore. It was never meant to be committed or published. But in this particular public-preview workflow, it had to remain available to the deployment and Docker build process.

The example environment file stayed in the repository with empty placeholder values, while the real secrets remained local.

The one invisible character that broke the model call

Once the hosted runtime could see the credentials, the diagnostic output showed something unexpected:

"model_id": "openai-gpt-oss-120b\r"

The model ID looked correct in a text editor.

It looked correct in a normal console print.

But it contained a trailing carriage-return character.

To the inference service, these were not the same model:

openai-gpt-oss-120b

and:

openai-gpt-oss-120b\r

The second value did not match a valid model identifier.

This is the kind of bug that can consume far more time than its size suggests.

The project was being developed in Windows, built into a Linux container, packaged by another CLI, and executed in a hosted Linux environment. That meant line endings and environment parsing could cross several boundaries.

PowerShell was used to inspect the exact character codes and verify that the cleaned model ID contained 19 characters.

The Python application was then hardened so environment values were sanitized rather than trusted blindly. It removed:

• Leading and trailing whitespace
• Actual carriage-return characters
• Newline characters
• Literal \r and \n suffixes

The .env loader was also changed to override stale runtime values. This mattered because a contaminated value could already exist in the process environment before python-dotenv loaded the corrected file.

After those changes, the hosted diagnostic finally reported:

"model_id": "openai-gpt-oss-120b",
"model_id_length": 19

The model call then succeeded.

The final result

The working hosted request described an AI agent that creates rack-deployment checklists, with spiky traffic and no requirement for a custom model or operating-system control.

LangGraph selected:

"route": "serverless"

The model then produced the structured recommendation.

It explained that Docker Compose belonged in the local development workflow, the LangGraph application belonged on DigitalOcean’s hosted ADK runtime, and the model call belonged on Serverless Inference.

It also described the point at which the workload should be reevaluated for Dedicated Inference: not according to a made-up threshold, but when real measurements of usage, latency, throughput, concurrency, and cost supported that move.

That was exactly the behavior the project was intended to demonstrate.

What surprised me about DigitalOcean

The most surprising part was how little infrastructure I had to configure directly.

I did not create a permanent virtual machine for the agent.

I did not manually configure a reverse proxy.

I did not create a load balancer.

I did not install a production process manager.

I did not manually expose a container port to the internet.

Once the application contract was correct, the ADK handled the hosted runtime and returned a working HTTPS endpoint.

The Gradient CLI also performed pre-deployment validation. It checked that the project could import successfully, that dependencies could be installed, and that the expected ADK entrypoint was present before creating the release.

The control plane organized deployments into managed workspaces and exposed release history and runtime logs. DigitalOcean recommends logs and agent metrics for monitoring ADK applications, and the CLI also provides a gradient agent logs command for troubleshooting.

That developer experience was impressively direct.

The complications were mostly at the edges:

• Which configuration value takes precedence?
• Which files are included in which build stage?
• What does the hosted runtime inherit?
• How are Windows line endings interpreted inside Linux?
• How should secrets be handled while the product remains in preview?

Those are real issues, but they are also the kinds of issues one expects to become smoother as a young developer platform matures.

DigitalOcean’s broader Gradient platform became generally available in July 2025, while the custom-code ADK used for this project remains in public preview. That distinction matters. The larger AI platform is established, but this particular deployment workflow is still evolving.

Why I would use this approach again

There are easier ways to call a language model.

A single Python script could send a request to an API and print the response.

But that would miss the point of an agentic application.

The value came from combining several layers:

The Takeaway…

Each layer solved a different problem.

LangGraph made the decision flow explicit.

The model made the result readable.

Validation made the result safer.

Docker made development repeatable.

The ADK made deployment accessible.

Serverless Inference made the model available without requiring a dedicated GPU deployment on day one.

And the routing model preserved a path toward Dedicated Inference or GPU infrastructure when the application’s requirements justified it.

The larger hardware story

The agent ends by identifying a hardware scale-up trigger because software architecture does not remain abstract forever.

An application may begin with occasional calls to a serverless model.

Then usage becomes consistent.

Latency expectations tighten.

Context grows.

Concurrency increases.

A custom model becomes necessary.

Eventually, the conversation moves from API calls to GPU capacity.

From GPU capacity, it moves to memory.

From memory, it moves to model parallelism, storage throughput, networking, power, and cooling.

The progression can look something like this:

Prototype or spiky demand
        |
        v
Serverless Inference
        |
        v
Steady production workload
        |
        v
Dedicated Inference
        |
        v
Custom runtime or OS control
        |
        v
GPU Droplet
        |
        v
Multi-GPU and multi-node infrastructure
        |
        v
Rack-scale power, networking, storage, and cooling

That is the deeper purpose of the DoggyDish Deployment Retriever.

It is not just recommending a cloud product.

It is demonstrating how an agentic application can begin with a simple managed service while retaining a logical path toward dedicated, higher-performance infrastructure.

The application should not begin overengineered.

But it should know what growth looks like.

Final takeaway

DigitalOcean made the final hosting step simpler than I expected.

The command-line workflow was compact, the deployment was validated automatically, the hosted endpoint was created without manually building a production server, and the same platform provided access to the model used by the agent.

The challenges were not primarily about provisioning infrastructure.

They were about controlling the boundaries between tools:

• The boundary between deterministic logic and probabilistic generation
• The boundary between the source archive and Docker build context
• The boundary between local configuration and hosted environment variables
• The boundary between Windows text formatting and Linux execution
• The boundary between a plausible recommendation and a validated one

Those boundaries are where many real agent projects succeed or fail.

The final code is not especially large. The graph is not especially complicated. The deployment command is only a few words.

But behind that simplicity is a more important architecture:

Use code to make the decision, use the model to explain it, validate what the model returns, and choose infrastructure that can grow with the application.

The detailed implementation and follow-along build will be published separately on DoggyDish.