Part 2: Production Hardening a Full-Stack AI Agent on Cloudflare

Kristopher Persad
Jun 12
10 min read

Publication Scope and IP Boundary: This reference architecture is a personal KrisperTech publication based on public Cloudflare services, publicly available documentation, and a personal demo implementation. It is not an official Cloudflare product document. It does not disclose Cloudflare internal material, customer architecture, roadmap information, proprietary field assets, internal enablement content, source code, private configuration, secrets, or customer-specific implementation patterns. The purpose of this article is to explain a reusable architecture pattern for deploying a secure AI assistant using publicly available Cloudflare developer and security services.

The first phase of the KrisperTech AI assistant reference architecture proved that a full-stack AI agent could be built on Cloudflare.

The assistant used Cloudflare Pages for the frontend, Turnstile for human verification, Workers and the Agents SDK for the agent runtime, Durable Objects for session state, Workers AI for inference, AI Gateway for observability and control, Browser Run for content retrieval, and Cloudflare Access to protect the MCP tool layer. [1][2][3][4][5][6][7][8]

That proved the system could work.

This reference update is focused on a different question: What had to change for the architecture to be more reliable, governable, and production-ready?

That distinction matters.

A working AI demo can respond to prompts, stream answers, and call a model. A hardened AI agent has to do more. It needs reliable tool execution, enforceable governance controls, predictable retrieval, and clear boundaries between the user, agent, model, tools, and data sources.

The first version of KrisperBot worked end to end, but real testing exposed three (3) architectural issues:

The model needed to reliably support structured tool use.
The inference path needed to support the intended AI Gateway governance controls.
The retrieval layer needed to discover article content reliably instead of depending on general link crawling.

These were not cosmetic changes. They were architecture changes made to support production hardening.

The Architecture Being Hardened

The original architecture was intentionally full stack.

It included:

A browser-based chat interface on Cloudflare Pages
Human verification using Turnstile
A serverless AI agent running on Workers
Stateful session handling with Durable Objects
LLM inference through Workers AI
AI Gateway for logging, token visibility, cost tracking, and policy controls
MCP-based tool exposure
Cloudflare Access protecting the MCP tool server
Browser Run for rendering live website content
KrisperTech.com as the initial public knowledge source

This was not simply a chatbot connected to a model. It was an application architecture with frontend delivery, runtime logic, inference, tools, access control, retrieval, and observability.

The architecture was functional. The question was whether each layer behaved correctly under realistic testing.

That is where the production hardening work began.

Why Production Hardening Required Architecture Changes

Production hardening is not always about adding more controls.

Sometimes it is about proving that the controls, paths, and dependencies already in the design behave the way the architecture assumes they do.

AI agents make this especially important because they are not a single component. They are a chain of components.

The model has to produce the right kind of output. The agent runtime has to parse and act on it. The inference request has to pass through the intended control layer. The tool server has to be protected. The retrieval layer has to return the right content. The frontend has to keep the experience usable while all of that happens.

If one layer behaves differently than expected, the assistant may still appear to work while failing an important production requirement.

Here we identified three (3) places where that was true.

Hardening Decision 1: Use a Model That Fits the Agentic Workload

The first issue appeared during tool-calling tests.

The assistant needed to answer questions about KrisperTech.com. To do that, the agent had to decide when website content was required, call the correct tool, receive the result, and continue reasoning with that returned context.

That requires structured tool use.

In early testing, the model could understand that a tool should be called, but the output was not reliable enough for the agent runtime. Instead of consistently producing a structured tool invocation, the model could describe the tool call as plain text.

For a human reader, that may look close enough.

For an agent runtime, it is a failure.

A text description of a tool call is not the same thing as an executable tool call.

The architecture depended on the model producing structured output that the agent could parse and execute. Once that requirement was tested directly, model selection became an architectural decision, not just a performance preference.

The assistant was moved to Kimi K2.6 on Workers AI because the workload required stronger support for multi-turn tool calling and structured outputs. Cloudflare’s Workers AI model card describes Kimi K2.6 as supporting multi-turn tool calling and structured outputs for agentic workloads. [3]

The hardening lesson is simple: For AI agents, model selection must be tested against the behavior the architecture requires.

A model that performs well at general reasoning may still be a poor fit for a tool-heavy system if it cannot reliably produce structured tool calls.

For production agent architectures, teams should test:

Whether the model selects tools correctly
Whether it produces valid structured tool calls
Whether it handles tool results correctly
Whether it continues reasoning after tool execution
Whether it avoids tool use when no tool is needed
Whether behavior remains consistent across multi-turn conversations

The model is not just an interchangeable backend. It is part of the architecture.

Hardening Decision 2: Put Governance Controls on the Actual Inference Path

The second issue was about governance.

The architecture included AI Gateway because production AI systems need visibility and control. AI Gateway can provide logging, analytics, token visibility, cost tracking, rate limiting, caching, DLP, and guardrail capabilities depending on configuration and request path. [4][5]

But having a gateway in the architecture is not the same thing as proving that the assistant’s real inference traffic is being evaluated by the intended controls.

That was the lesson: A control only matters if the traffic actually traverses the control point.

During testing, the assistant used synthetic sensitive-data prompts to validate whether DLP policies were being applied as expected. The testing showed that the inference path needed to be adjusted so the request and response bodies moved through the intended AI Gateway enforcement path.

This was not merely a policy change.

It was an architecture change.

The question was not “is there a DLP policy configured?”

The question was:

Which path does the agent use to reach the model?
Does that path pass through the AI Gateway policy layer?
Are request and response bodies evaluated?
What appears in the logs?
What happens when a request is blocked?
Does the application handle that failure cleanly?

In this phase, the inference path was changed to use the AI Gateway REST API path for the governance behavior being tested.

The broader hardening principle is that AI governance must be validated on the actual runtime path.

This matters because AI applications may have more than one way to reach inference:

Native bindings
REST API calls
SDK abstractions
Gateway-mediated requests
Provider-specific endpoints
Custom proxy paths

Those paths may not expose the same logging, inspection, or enforcement behavior.

For production systems, the inference path should be documented and tested like any other security-sensitive request path.

Hardening Decision 3: Make Retrieval Deterministic

The third issue involved content retrieval.

The assistant was designed to answer questions about KrisperTech.com. The original retrieval strategy used general link crawling from the website.

That seemed reasonable at first.

Start from the homepage, follow links, discover pages, and extract content, but testing showed that this approach was not reliable enough for the site structure.

KrisperTech.com is built on Wix, which means navigation and content discovery can involve JavaScript-heavy rendering and single-page application behavior. General link crawling could spend its page budget on homepage content, navigation paths, category pages, tag pages, or structural URLs before reaching the actual blog articles.

The result was that the assistant could retrieve some website content, but not consistently discover the article content it needed.

That created a grounding problem.

If the retrieval layer does not find the right content, the model cannot reliably answer from that content.

The fix was to separate discovery from extraction.

Instead of relying on a general link crawl to find articles, the retrieval strategy changed to use the website sitemap as the authoritative source of article URLs. Browser Run could then be used where it added the most value: rendering and extracting content from individual JavaScript-rendered pages. Browser Run documentation supports crawling web content and provides guidance around sitemaps and site access patterns. [6][7]

This was a small change with a large architectural impact.

The hardened retrieval pattern became:

Use the sitemap for deterministic article discovery.
Filter for the relevant article URLs.
Use Browser Run for page rendering and content extraction.
Return cleaner grounding content to the agent.

The broader lesson is: Grounding quality depends on retrieval architecture.

For public websites, a sitemap-first approach may be more reliable than open-ended link crawling.

For enterprise systems, the equivalent might be:

Querying a document index
Reading from object storage
Searching a vector index
Calling a knowledge-base API
Retrieving content from a database
Using metadata filters before semantic search

The retrieval strategy should match the structure of the knowledge source.

Do not assume the agent can find the right content simply because that content exists somewhere.

What Stayed the Same

Hardening does not mean changing every component.

Some parts of the original architecture stayed because they continued to support the design goals.

Streaming stayed enabled

Streaming remained important to the user experience.

When a user asks a question that requires retrieval, reasoning, and generation, a blank screen creates uncertainty. Streaming makes the system feel responsive even when the backend is doing more work.

That design choice does involve tradeoffs, because some AI control features may behave differently with streaming than with non-streaming responses. The decision was to preserve the user experience while validating which controls applied to the selected path.

MCP remained separate from the agent runtime

The MCP server stayed as a separate tool layer.

That separation is valuable.

The agent should not casually expose every tool directly inside the same runtime. A separate tool layer makes it easier to protect, monitor, and evolve tool access independently.

In this architecture, Cloudflare Access protects the MCP server, and service tokens provide a machine authentication path for the agent. Cloudflare’s Access documentation describes service tokens as static credential pairs for automated workflows where no user is present, and Cloudflare also documents patterns for securing MCP servers with Access. [8][9]

That reinforces a key architecture point: Agent tools are privileged capabilities and should sit behind intentional access boundaries.

Durable Objects stayed for per-session state

Durable Objects remained part of the design because agent sessions benefit from stateful coordination. Cloudflare describes Durable Objects as a building block for stateful applications, including applications that need coordination among clients and real-time interactions. [2]

For this assistant, the state model supports conversation continuity and per-session behavior. A stateless version may be simpler, but it would not demonstrate the same agent pattern.

Before and After

The hardening changes were targeted.

The goal was not to rebuild the assistant. The goal was to align the architecture with production behavior.

Area	Initial Pattern	Hardened Pattern	Reason
Model behavior	General model selection	Model selected for agentic tool use	The agent needed reliable structured tool calls
Inference path	Working inference path	AI Gateway REST API path for governance validation	Controls needed to apply on the actual request path
Retrieval	General link crawling	Sitemap-based discovery plus Browser Run extraction	The agent needed deterministic article discovery
Tool layer	MCP server	MCP server retained	Separation supported access control and auditability
User experience	Streaming responses	Streaming retained	Streaming preserved responsiveness
State	Durable Object sessions	Durable Object sessions retained	Per-session state remained appropriate

The important point is not that Phase 1 was wrong.

The important point is that Phase 2 tested the assumptions Phase 1 depended on.

That is what production hardening is supposed to do.

Practical Lessons for AI Agent Architecture

Phase 2 produced a few lessons that apply beyond this implementation.

1. Test the model against the architecture

Do not test only whether the model can answer questions.

Test whether it can perform the specific behaviors your architecture requires: tool use, structured output, multi-turn continuation, and safe handling of tool results.

2. Validate the governance path

Do not assume that logging, DLP, guardrails, or rate limits apply simply because they are configured.

Confirm that the application’s real inference path passes through the control layer where those policies are evaluated.

3. Treat retrieval as part of the trust model

Grounded AI depends on retrieval quality.

If retrieval is unreliable, the model may produce weak or incomplete answers even when the knowledge source contains the right information.

4. Separate tools from the agent

Tool use is where AI agents become operationally powerful.

A separate tool layer creates a cleaner place to apply access control, authentication, logging, and input validation.

5. Preserve user experience deliberately

Streaming, progress behavior, and response handling are not cosmetic choices. They influence whether the assistant feels reliable during longer retrieval and reasoning tasks.

Security Takeaways

AI security is not only about blocking bad prompts.

It is about designing the system so the agent operates within known boundaries.

In this architecture, those boundaries include:

Turnstile before the assistant consumes AI resources
Access protecting the MCP tool server
Service tokens for machine-to-machine authentication
AI Gateway for inference visibility and governance
Sitemap-based discovery to constrain content retrieval
Browser Run used intentionally for rendering and extraction
Durable Objects for isolated per-session behavior

No single control makes the architecture production-ready.

The value comes from the combination of controls and the validation that they apply where expected.

Conclusion

Phase 1 proved the assistant could work.

Phase 2 made the architecture stronger.

The most important lesson was that production hardening is architectural.

The model had to match the tool-calling workload. The inference path had to support the intended governance controls. The retrieval layer had to reliably discover the right content. The tool layer had to remain protected. The user experience had to stay responsive.

That is the difference between building an AI demo and hardening an AI agent architecture.

A chatbot responds.

An AI agent operates inside a system.

Once an agent can use tools, retrieve content, and interact with governed services, the architecture around the model becomes just as important as the model itself.

The future of enterprise AI will not be defined only by better models. It will be defined by better architectures around those models: architectures that govern inference, constrain tool use, retrieve trusted content, protect access paths, observe behavior, and deliver a usable experience.

That is what this Phase 2 hardening effort was designed to demonstrate.