Building Production Agents on Amazon Bedrock: From PoC to Production

Introduction: The Demo-to-Production Gap

AI demos are easy. You wire up an API call to Claude or GPT, pipe in a user question, and get back a plausible-sounding answer. The demo works. Leadership is impressed. The team gets a green light to "productionize it." And then everything stalls.

The gap between a notebook prototype and a reliable, observable, safe AI agent is where most teams lose three to six months. The prototype has no guardrails, no evaluation framework, no cost controls, and no observability. It hallucinates freely and nobody notices because nobody is measuring. It calls tools when it should not and fails silently when it should. It works on the ten examples you tested and breaks on the ten thousand your customers will send.

Amazon Bedrock Agents closes a significant portion of that gap. It provides managed infrastructure, built-in tool use through action groups, retrieval-augmented generation through Knowledge Bases, and content safety through Guardrails. But managed infrastructure does not mean zero design work. You still need to architect the agent properly, define the right tool boundaries, build an evaluation harness, and instrument everything for production observability. This guide walks through the full path from concept to a production-grade Bedrock Agent.

What Bedrock Agents Are

A Bedrock Agent is an orchestration layer that sits between your users and a foundation model. When a user sends a request, the agent reasons about what the user needs, decides which tools to call (if any), retrieves relevant context from your data, and returns a grounded response. It is not just a prompt wrapper — it is a stateful reasoning loop that can chain multiple tool calls and knowledge retrievals to answer complex questions.

Under the hood, the agent uses a foundation model to power its reasoning. Bedrock supports Claude (Anthropic), Titan (Amazon), Llama (Meta), and Mistral as foundation models for agents. You choose the model at agent creation time based on your requirements for quality, latency, and cost. Claude models tend to excel at complex reasoning and tool use. Titan is cost-effective for simpler tasks. Llama and Mistral offer open-weight alternatives for teams with specific compliance needs.

The critical point is that Bedrock Agents are fully managed. You do not provision GPUs. You do not deploy inference endpoints. You do not manage model serving infrastructure. You configure the agent declaratively — which model, which tools, which knowledge sources, which guardrails — and Bedrock handles the rest. This is not a trivial benefit. Teams that self-host agent infrastructure typically spend 40-60% of their engineering time on infrastructure concerns rather than agent logic.

Architecture Overview

A production Bedrock Agent consists of four core components, each with a distinct responsibility:

• Agent — The orchestrator. It receives the user request, reasons about it using the foundation model, decides which tools and knowledge sources to invoke, and assembles the final response. The agent manages conversation state and can chain multiple reasoning steps.
• Knowledge Bases — Your data, vectorized and searchable. Documents uploaded to S3 are chunked, embedded, and stored in a vector database (OpenSearch Serverless or Aurora PostgreSQL with pgvector). When the agent needs context, it performs a semantic search against the Knowledge Base and injects the relevant chunks into its reasoning context.
• Action Groups — External capabilities exposed as Lambda functions. These let the agent interact with the outside world: query a database, call a third-party API, trigger a workflow, or perform calculations. Each action group is defined by an OpenAPI schema that tells the agent what the tool does, what parameters it accepts, and what it returns.
• Guardrails — Content filters, PII redaction, denied topics, and grounding checks applied to both inputs and outputs. Guardrails are the safety net that prevents the agent from generating harmful content, leaking sensitive data, or straying outside its intended scope.

Setting Up a Knowledge Base

The Knowledge Base is where your agent gets its grounding data. Without it, the agent can only rely on the foundation model's training data, which is stale and generic. With a Knowledge Base, the agent can answer questions about your specific products, policies, documentation, and internal data.

The setup flow is straightforward: upload your documents to an S3 bucket, create a Knowledge Base that points to that bucket, choose an embedding model (Titan Embeddings v2 is the default and handles most use cases well), and choose a vector store. OpenSearch Serverless is the default and requires no capacity planning. Aurora PostgreSQL with pgvector is a good choice if you already run Aurora and want to avoid an additional service.

The chunking strategy is the single most important decision you will make when configuring a Knowledge Base. Bedrock offers fixed-size chunking (default), hierarchical chunking, and semantic chunking. Fixed-size chunking with a 300-token chunk and 20% overlap is a reasonable starting point. Semantic chunking produces better results for documents with clear section boundaries but costs more due to the additional embedding calls.

# Step 1: Create the Knowledge Base
aws bedrock-agent create-knowledge-base \
  --name "product-docs-kb" \
  --role-arn "arn:aws:iam::123456789012:role/BedrockKBRole" \
  --knowledge-base-configuration '{
    "type": "VECTOR",
    "vectorKnowledgeBaseConfiguration": {
      "embeddingModelArn": "arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v2:0"
    }
  }' \
  --storage-configuration '{
    "type": "OPENSEARCH_SERVERLESS",
    "opensearchServerlessConfiguration": {
      "collectionArn": "arn:aws:aoss:us-east-1:123456789012:collection/abc123",
      "vectorIndexName": "product-docs-index",
      "fieldMapping": {
        "vectorField": "embedding",
        "textField": "text",
        "metadataField": "metadata"
      }
    }
  }'

# Step 2: Add an S3 data source to the Knowledge Base
aws bedrock-agent create-data-source \
  --knowledge-base-id "KB_ID_FROM_STEP_1" \
  --name "product-docs-s3" \
  --data-source-configuration '{
    "type": "S3",
    "s3Configuration": {
      "bucketArn": "arn:aws:s3:::my-product-docs-bucket",
      "inclusionPrefixes": ["docs/", "faqs/"]
    }
  }' \
  --vector-ingestion-configuration '{
    "chunkingConfiguration": {
      "chunkingStrategy": "FIXED_SIZE",
      "fixedSizeChunkingConfiguration": {
        "maxTokens": 300,
        "overlapPercentage": 20
      }
    }
  }'

# Step 3: Trigger ingestion
aws bedrock-agent start-ingestion-job \
  --knowledge-base-id "KB_ID_FROM_STEP_1" \
  --data-source-id "DS_ID_FROM_STEP_2"

After ingestion completes, test the Knowledge Base independently before attaching it to an agent. Use the retrieve API to send test queries and verify that the returned chunks are relevant. If the chunks are consistently off-target, adjust your chunking strategy before going further. Bad retrieval cannot be fixed by a smarter agent.

Creating an Action Group

Action groups give your agent the ability to do things, not just know things. Each action group is backed by a Lambda function and described by an OpenAPI schema. The agent reads the schema to understand what the tool does, decides when to call it based on the user's request, and passes the appropriate parameters. The Lambda function executes the logic and returns a result that the agent incorporates into its response.

A common first action group is an order lookup. The agent receives a customer question like "Where is my order 12345?", recognizes it needs order data, calls the LookupOrder action group with the order ID, and returns the status to the customer. Here is the Lambda handler:

import json
import boto3

dynamodb = boto3.resource("dynamodb")
orders_table = dynamodb.Table("Orders")


def lambda_handler(event, context):
    """Handle Bedrock Agent action group invocation."""

    # Extract the action and parameters from the Bedrock Agent event
    action_group = event.get("actionGroup")
    api_path = event.get("apiPath")
    parameters = event.get("parameters", [])

    # Build a parameter dict from the Bedrock Agent format
    params = {p["name"]: p["value"] for p in parameters}

    if api_path == "/lookup-order":
        order_id = params.get("orderId")
        if not order_id:
            return _build_response(event, 400, {"error": "orderId is required"})

        # Query DynamoDB for the order
        response = orders_table.get_item(Key={"orderId": order_id})
        item = response.get("Item")

        if not item:
            return _build_response(event, 404, {"error": f"Order {order_id} not found"})

        return _build_response(event, 200, {
            "orderId": item["orderId"],
            "status": item["status"],
            "estimatedDelivery": item.get("estimatedDelivery", "Unknown"),
            "trackingNumber": item.get("trackingNumber", "Not yet assigned"),
        })

    return _build_response(event, 400, {"error": f"Unknown API path: {api_path}"})


def _build_response(event, status_code, body):
    """Build a response in the format Bedrock Agent expects."""
    return {
        "messageVersion": "1.0",
        "response": {
            "actionGroup": event.get("actionGroup"),
            "apiPath": event.get("apiPath"),
            "httpMethod": event.get("httpMethod"),
            "httpStatusCode": status_code,
            "responseBody": {
                "application/json": {
                    "body": json.dumps(body)
                }
            },
        },
    }

The response format is specific to Bedrock Agents. The agent expects a messageVersion, the action group name echoed back, and a structured response body. Getting this format wrong is the most common cause of action group failures — the Lambda succeeds but the agent cannot parse the result.

Defining the Agent

With the Knowledge Base and action group ready, you can create the agent itself. The agent definition ties everything together: which foundation model to use, what instructions to follow, which knowledge sources and tools are available, and which guardrails to apply.

The agent instruction is critical. This is not a casual system prompt — it is the operating contract that governs how the agent behaves. Be explicit about what the agent should and should not do, when to use tools versus when to answer from knowledge, and how to handle ambiguous requests.

# Create the agent
aws bedrock-agent create-agent \
  --agent-name "customer-support-agent" \
  --foundation-model "anthropic.claude-sonnet-4-20250514" \
  --agent-resource-role-arn "arn:aws:iam::123456789012:role/BedrockAgentRole" \
  --instruction "You are a customer support agent for Acme Corp. Your role is to
help customers with order inquiries, product questions, and return requests.

Rules:
- Always look up order information using the LookupOrder tool before answering
  order-related questions. Never guess order status.
- Use the Knowledge Base for product specifications, return policies, and FAQs.
- If a customer asks about pricing, promotions, or account billing, escalate to
  a human agent. Do not attempt to answer.
- Never disclose internal processes, employee names, or system details.
- Keep responses concise and professional." \
  --idle-session-ttl-in-seconds 600

# Associate the Knowledge Base
aws bedrock-agent associate-agent-knowledge-base \
  --agent-id "AGENT_ID" \
  --knowledge-base-id "KB_ID" \
  --description "Product documentation, FAQs, and return policies" \
  --knowledge-base-state "ENABLED"

# Create an action group (references the Lambda and OpenAPI schema)
aws bedrock-agent create-agent-action-group \
  --agent-id "AGENT_ID" \
  --action-group-name "OrderManagement" \
  --action-group-executor '{"lambda": "arn:aws:lambda:us-east-1:123456789012:function:order-lookup"}' \
  --api-schema '{"s3": {"s3BucketName": "my-schemas-bucket", "s3ObjectKey": "order-api.json"}}'

# Prepare the agent (compiles configuration)
aws bedrock-agent prepare-agent --agent-id "AGENT_ID"

# Create a versioned alias for production use
aws bedrock-agent create-agent-alias \
  --agent-id "AGENT_ID" \
  --agent-alias-name "production"

# The alias gives you a stable endpoint. Update the alias to point
# to new agent versions as you iterate — no client-side changes needed.

The alias system is worth understanding. Each time you call prepare-agent, Bedrock creates a new version of the agent. The alias points to a specific version. In production, your application always invokes the alias, not the agent directly. When you want to deploy a new agent configuration, you prepare a new version, test it, and then update the alias to point to the new version. This gives you safe rollbacks and blue-green deployment semantics for your agent.

Guardrails: The Non-Negotiable

If you ship an agent without Guardrails, your customers will find a jailbreak on day one. This is not hyperbole. Every production AI system that has launched without content safety controls has been publicly embarrassed within hours. A car dealership chatbot was tricked into offering a $1 car. A parcel delivery chatbot was convinced to write poetry criticizing the company. These are not sophisticated attacks — they are basic prompt injection that Guardrails prevent.

Bedrock Guardrails give you five layers of protection:

• Content filters — Block or flag content across hate, violence, sexual, insults, and misconduct categories. Each category has configurable strength (NONE, LOW, MEDIUM, HIGH).
• Denied topics — Define topics the agent must refuse to discuss. Examples: competitor comparisons, medical advice, legal advice, investment recommendations.
• Word filters — Block specific words or phrases, including profanity and custom terms (competitor names, internal project codenames).
• PII redaction — Automatically detect and mask personally identifiable information in both inputs and outputs. Supports SSN, credit card numbers, phone numbers, email addresses, and more. You can choose to block the entire request or anonymize the PII inline.
• Grounding check — Verifies that the agent's response is actually supported by the Knowledge Base context. If the agent generates a claim not backed by retrieved documents, the grounding check catches it. This is your primary defense against hallucination.

{
  "name": "customer-support-guardrail",
  "description": "Guardrail for customer-facing support agent",
  "contentPolicyConfig": {
    "filtersConfig": [
      { "type": "HATE",        "inputStrength": "HIGH", "outputStrength": "HIGH" },
      { "type": "VIOLENCE",    "inputStrength": "HIGH", "outputStrength": "HIGH" },
      { "type": "SEXUAL",      "inputStrength": "HIGH", "outputStrength": "HIGH" },
      { "type": "INSULTS",     "inputStrength": "HIGH", "outputStrength": "HIGH" },
      { "type": "MISCONDUCT",  "inputStrength": "HIGH", "outputStrength": "HIGH" }
    ]
  },
  "topicPolicyConfig": {
    "topicsConfig": [
      {
        "name": "CompetitorDiscussion",
        "definition": "Any comparison with or mention of competitor products or services",
        "examples": ["How does your product compare to Zendesk?", "Is Acme better than Freshdesk?"],
        "type": "DENY"
      },
      {
        "name": "LegalAdvice",
        "definition": "Requests for legal opinions, contract interpretation, or liability guidance",
        "examples": ["Can I sue you for late delivery?", "What are my legal rights?"],
        "type": "DENY"
      }
    ]
  },
  "sensitiveInformationPolicyConfig": {
    "piiEntitiesConfig": [
      { "type": "US_SOCIAL_SECURITY_NUMBER", "action": "BLOCK" },
      { "type": "CREDIT_DEBIT_CARD_NUMBER",  "action": "ANONYMIZE" },
      { "type": "EMAIL",                      "action": "ANONYMIZE" },
      { "type": "PHONE",                      "action": "ANONYMIZE" }
    ]
  },
  "contextualGroundingPolicyConfig": {
    "filtersConfig": [
      { "type": "GROUNDING",  "threshold": 0.7 },
      { "type": "RELEVANCE",  "threshold": 0.7 }
    ]
  }
}

Create the guardrail with aws bedrock create-guardrail --cli-input-json file://guardrail-config.json and then associate it with your agent. The grounding threshold of 0.7 means the agent's response must be at least 70% supported by retrieved context. Start at 0.7 and tune based on your evaluation results — too high and the agent refuses to answer legitimate questions, too low and hallucinations slip through.

Invoking the Agent

With the agent configured, prepared, and aliased, you can invoke it from your application code. The agent returns a streaming response, which is important for user experience — users see the response building incrementally rather than waiting for the entire reasoning chain to complete.

import boto3
import uuid
import json

bedrock_agent_runtime = boto3.client("bedrock-agent-runtime")

AGENT_ID = "ABCDEF1234"
AGENT_ALIAS_ID = "PROD_ALIAS_ID"


def invoke_agent(user_message: str, session_id: str | None = None) -> dict:
    """Invoke a Bedrock Agent and return the response with citations."""

    session_id = session_id or str(uuid.uuid4())

    response = bedrock_agent_runtime.invoke_agent(
        agentId=AGENT_ID,
        agentAliasId=AGENT_ALIAS_ID,
        sessionId=session_id,
        inputText=user_message,
        enableTrace=True,  # Enable trace for observability
    )

    # Process the streaming response
    completion = ""
    citations = []
    trace_steps = []

    for event in response["completion"]:
        # Collect the text chunks
        if "chunk" in event:
            chunk = event["chunk"]
            completion += chunk["bytes"].decode("utf-8")

            # Extract citations from the chunk
            if "attribution" in chunk:
                for citation in chunk["attribution"].get("citations", []):
                    for ref in citation.get("retrievedReferences", []):
                        citations.append({
                            "text": ref["content"]["text"],
                            "source": ref["location"]["s3Location"]["uri"],
                        })

        # Collect trace information for debugging
        if "trace" in event:
            trace_data = event["trace"].get("trace", {})
            if "orchestrationTrace" in trace_data:
                trace_steps.append(trace_data["orchestrationTrace"])

    return {
        "answer": completion,
        "citations": citations,
        "session_id": session_id,
        "trace_steps": trace_steps,
    }


# Usage
result = invoke_agent("Where is my order ORD-98765?")
print(f"Answer: {result['answer']}")
print(f"Sources: {[c['source'] for c in result['citations']]}")
print(f"Reasoning steps: {len(result['trace_steps'])}")

The enableTrace=True flag is essential for production. The trace shows the full reasoning chain: what the agent decided to do, which tools it called, what parameters it passed, what the tools returned, and how it assembled the final answer. Without traces, debugging agent failures is guesswork.

Evaluation Before Production

"It looks right" is not an evaluation strategy. Before any agent goes to production, you need an eval harness — a repeatable, automated process that measures agent quality across a representative dataset. Without it, you have no way to know whether a configuration change improved or degraded the agent, and no way to detect quality regression over time.

The evaluation dataset should contain at minimum 50-100 test cases, each structured as a tuple of (question, expected_answer, expected_sources). The questions should cover the full range of your agent's expected use cases, including edge cases and adversarial inputs. Measure four dimensions:

• Answer correctness — Does the agent's response contain the right information? Use an LLM-as-judge approach or exact match for factual questions.
• Faithfulness — Is the response grounded in the retrieved context? An answer can be correct but unfaithful if the agent used its parametric knowledge instead of the Knowledge Base.
• Latency — End-to-end response time including tool calls and Knowledge Base retrievals. Set a P95 latency target (e.g., under 8 seconds for customer support).
• Cost per invocation — Total token usage (input + output) plus KB query costs and Lambda invocation costs. Track this per test case to identify expensive query patterns.

Bedrock has a built-in Model Evaluation feature that supports human evaluation workflows and automated metrics. For continuous evaluation, build a lightweight harness that runs on every agent configuration change:

import json
import time
from dataclasses import dataclass


@dataclass
class EvalCase:
    question: str
    expected_answer: str
    expected_sources: list[str]
    category: str


def load_eval_dataset(path: str) -> list[EvalCase]:
    with open(path) as f:
        data = json.load(f)
    return [EvalCase(**case) for case in data]


def evaluate_agent(eval_cases: list[EvalCase]) -> dict:
    """Run the agent against all eval cases and compute metrics."""
    results = []

    for case in eval_cases:
        start = time.time()
        result = invoke_agent(case.question)  # from invoke_agent.py
        latency = time.time() - start

        # Check if expected answer content is present
        correct = case.expected_answer.lower() in result["answer"].lower()

        # Check if citations reference expected sources
        cited_sources = {c["source"] for c in result["citations"]}
        expected = set(case.expected_sources)
        faithfulness = len(cited_sources & expected) / max(len(expected), 1)

        results.append({
            "question": case.question,
            "category": case.category,
            "correct": correct,
            "faithfulness": faithfulness,
            "latency_seconds": latency,
            "num_trace_steps": len(result["trace_steps"]),
        })

    # Aggregate metrics
    total = len(results)
    return {
        "accuracy": sum(r["correct"] for r in results) / total,
        "avg_faithfulness": sum(r["faithfulness"] for r in results) / total,
        "p50_latency": sorted(r["latency_seconds"] for r in results)[total // 2],
        "p95_latency": sorted(r["latency_seconds"] for r in results)[int(total * 0.95)],
        "avg_trace_steps": sum(r["num_trace_steps"] for r in results) / total,
        "results": results,
    }


# Run evaluation
cases = load_eval_dataset("eval_dataset.json")
metrics = evaluate_agent(cases)
print(f"Accuracy: {metrics['accuracy']:.1%}")
print(f"Faithfulness: {metrics['avg_faithfulness']:.1%}")
print(f"P95 Latency: {metrics['p95_latency']:.1f}s")

Run this eval on every pull request that touches agent configuration. If accuracy drops below your threshold (aim for 85% or higher), the PR does not merge. This is the same discipline you apply to unit tests — the eval harness is your test suite for agent quality.

Observability in Production

An agent in production without observability is a liability. You need to know when it is slow, when it is wrong, when it is expensive, and when it is being abused. Bedrock Agents emit metrics and logs to CloudWatch, but you need to go beyond the defaults to get actionable visibility.

CloudWatch Metrics

Bedrock publishes invocation count, latency (P50, P90, P99), throttle count, and error count as CloudWatch metrics. Set alarms on all of these. A sudden spike in latency often indicates that the agent is making more tool calls than expected, which also drives up cost. A spike in throttles means you have hit your account-level throughput limit and need to request a quota increase.

Agent Trace Logs

When you enable trace logging, every invocation produces a detailed log showing the agent's full reasoning chain. The trace includes: the initial prompt sent to the FM, the FM's reasoning about which tools to call, the tool invocations with parameters and responses, the Knowledge Base queries and retrieved chunks, and the final response assembly. Stream these traces to CloudWatch Logs and set up a Logs Insights query to surface patterns:

Cost Management

Bedrock pricing is per-token for the foundation model, per-query for Knowledge Base retrievals, and per-invocation for action group Lambda functions. This seems straightforward until you realize that a single user question can trigger multiple reasoning loops, each consuming tokens. A complex question that requires three tool calls and two Knowledge Base lookups can cost 5-10x more than a simple factual question.

The token cost trap is the agent trace itself. Each reasoning loop sends the full conversation context plus tool results back to the foundation model. By the third or fourth loop, the input token count can be substantial. A single complex invocation with Claude Sonnet and four tool calls can cost $0.05-0.15 in tokens alone. At scale, this adds up fast.

Strategies for controlling cost:

• Set max iterations — Limit the number of reasoning loops the agent can take. The default is generous. For most customer support agents, 3-4 iterations is sufficient.
• Use a smaller model for routing — If 60% of your queries are simple FAQ lookups, route them to a cheaper model (Haiku or Titan) and reserve the expensive model (Sonnet or Opus) for complex multi-step queries.
• Cache Knowledge Base results — If multiple users ask the same question within a short window, cache the retrieval results in ElastiCache or DynamoDB to avoid redundant vector searches.
• Optimize chunk size — Smaller chunks mean more chunks retrieved per query, which means more input tokens. Find the minimum number of chunks that maintains answer quality.
• Monitor cost per conversation — Set a per-session cost cap. If a conversation exceeds the cap, gracefully escalate to a human agent rather than letting the AI agent continue burning tokens on a query it cannot resolve.

Common Pitfalls

After helping multiple teams deploy Bedrock Agents, these are the failure modes we see most frequently:

From PoC to Production-Grade Agent

Bedrock Agents remove the undifferentiated heavy lifting of agent infrastructure: model serving, tool orchestration, retrieval pipelines, and content safety. But the service is a platform, not a solution. The quality of your agent depends on the quality of your Knowledge Base, the design of your action groups, the rigor of your guardrails, the discipline of your evaluation process, and the depth of your observability.

The teams that succeed with AI agents treat them like any other production system. They have test suites (eval harnesses). They have monitoring and alerting (CloudWatch metrics and traces). They have deployment processes (agent versioning and aliases). They have cost controls (iteration limits and per-session caps). And they have safety controls (Guardrails with grounding checks). Skip any one of these and you are shipping a prototype, not a product.