Building a Secure FAQ Chatbot Using Amazon Bedrock Knowledge Bases: A Guide and Lessons Learned

Imagine you’re on the IT Help Desk team, and every day is a scavenger hunt — but instead of looking for gold, you’re sifting through elusive SOPs, FAQs, and policy documents. Support agents waste time searching for company information. That’s the burden CloudNova, a fictional company for the purposes of this project, faces, with its support agents wasting valuable time searching for answers across multiple company documents.

Public AI tools seemed like a natural solution, but they created other problems: they couldn’t access the company’s own documents, posed a risk to sensitive information, and occasionally provided inaccurate or entirely made-up answers.

The Solution? Amazon Bedrock Knowledge Bases — a secure, private, enterprise-ready solution that answers exclusively from your internal documents on AWS.

In this post, I will show you how to create a secure document-aware FAQ chatbot using Amazon Bedrock Knowledge Bases. Along the way, I’ll discuss what I learned from this project, the challenges I encountered, and the insights I gained. In the end, you will have a complete chatbot and a better understanding of AWS capabilities.

The Alternative: A Safe, Enterprise-Grade FAQ Chatbot

This project aims to build a chatbot tailored to CloudNova’s needs. This chatbot will:

• Based exclusively on answers from internal documents.
• No hallucination or made-up information.
• Maintain all information safe and sound on AWS.
• With little coding and no ML training required.

The solution uses the following AWS services:

• Amazon Bedrock Knowledge Bases: Document retrieval and grounded answer generation.
• AWS Lambda Function URL: A lightweight, serverless backend.
• Amazon S3: To store docs and host the web app, use Amazon S3.
• AWS IAM: To maintain security.

What I Learned Throughout This Project

Before I get into the nuts and bolts, I just want to talk a bit about what I learned from doing this.

1. RAG Changes the Game

Retrieval-Augmented Generation is not only a concept — it’s widely applicable. I was impressed by how the Bedrock Knowledge Base found pertinent data across all my files and produced answers without hallucinating (a term for incorrect answers produced by AI). This is the right solution for companies that require dependable, document-centric AI solutions.

2. Serverless Is Just Better

AWS Lambda with Function URL made the backend wonderfully simple. There was no need to concern yourself with servers, scaling, or maintenance — AWS took care of that. This meant I could concentrate on the actual problem I was trying to solve, rather than the infrastructure.

3. You Can’t Compromise on Security

Creating IAM roles and policies is a tedious but necessary process. I focused on giving only what was minimally needed and added to the permission with every roadblock. Following the least-privilege principle.

4. Detail-Oriented Is Key

AWS is great, but it’s also a bit complicated. Just one small mistake, such as not syncing the Knowledge Base or missing a permission, and you’re stopped in your tracks. I learned to double-check everything and to take the AWS docs as my close friend.

5. Testing Builds Confidence

Testing is more than just finding bugs — it’s making sure your answers work the way they are supposed to. In-scope queries, out-of-scope queries, and edge cases running through my mind helped me be confident that my chatbot would work well in the environment.

How I created the FAQ chatbot Step-by-Step

Step 1: Environment Setup for FAQ Chatbot

Before building the FAQ chatbot, set up your AWS environment and permissions as follows:

Step 1: Enable Amazon Bedrock

1. Log in to the AWS Management Console.
2. Go to Amazon Bedrock.
3. If inactive, click Request Access and follow the steps.
4. Set your AWS Region to a supported region (e.g., us-east-1 or us-west-2).

Step 2: Create an S3 Bucket

1. In the Amazon S3 Console, click Create bucket.
2. Enter a unique bucket name (e.g., faq-chatbot-demo-docs).
3. Ensure Block Public Access is enabled.
4. Click Create bucket.

Step 3: Upload the SOP Document

1. Download the SOP file (e.g., CloudNova_SOP.pdf).
2. Open your S3 bucket.
3. Click Upload → Add files, select the SOP file, and upload it.

Step 4: Configure IAM Permissions

1. In IAM Console, go to Policies → Create policy.
2. Use the JSON tab to add permissions. Replace the bucket name with your own:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "S3AccessForRAG",
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:ListBucket"
      ],
      "Resource": [
        "arn:aws:s3:::faq-chatbot-demo-docs",
        "arn:aws:s3:::faq-chatbot-demo-docs/*"
      ]
    },
    {
      "Sid": "BedrockAccess",
      "Effect": "Allow",
      "Action": [
        "bedrock:CreateKnowledgeBase",
        "bedrock:Retrieve",
        "bedrock:RetrieveAndGenerate"
      ],
      "Resource": "*"
    },
    {
      "Sid": "PassRoleAccess",
      "Effect": "Allow",
      "Action": "iam:PassRole",
      "Resource": "*"
    }
  ]
}

3. Name the policy (e.g., BedrockRAGAccessPolicy) and create it.

Step 5: Create an IAM Role

1. Go to IAM Console → Roles → Create role.
2. Select AWS service → Amazon Bedrock Agentcore.
3. Attach the BedrockRAGAccessPolicy.
4. Name the role (e.g., BedrockRAGExecutionRole) and create it.

Step 6: Verify Setup

1. Confirm you can access the SOP file in S3.
2. Ensure your IAM role has the right permissions.
3. Verify Amazon Bedrock is active in your region.

Step 2: Create Knowledge Base in Bedrock

Now that our SOP document is uploaded to S3 and permissions are configured, it’s time to create a Knowledge Base in Amazon Bedrock.

Step 1: Open Amazon Bedrock Console

• Navigate to Amazon Bedrock.
• In the left navigation panel, click on “Knowledge bases”.
• Click “Create knowledge base.”

Step 2: Basic Configuration

• Enter a name for your Knowledge Base with vector store(e.g., faq-chatbot-kb).
• Choose: Create and use a new service role > Click Next.

Step 3: Connect Data Source to S3

• In the Configure data source panel:
• Data source name: enter something like cloudnova-sop-datasource.
• Data source location: leave this AWS account selected (unless your S3 bucket is in another account).
• S3 URI: Click Browse and select: s3://faq-chatbot-demo-docs/SOP-CloudNova.pdf
• Leave the other settings at their defaults and click Next.

Step 4: Configure Embeddings

Embeddings convert your document text into a vector representation, enabling fast and accurate retrieval of relevant content when users ask questions.

• In the “Select model” screen, choose the model provider: Select Amazon.
• Under Models, select the latest embedding model: Titan Text Embeddings V2.
• Click “Apply”.
• Under Vector store type, select Amazon S3 Vectors (Preview)
• The store will be automatically linked to your chosen Titan Embeddings model (e.g., Titan Embeddings G1 — Text).

Step 5: Review and Create

• Review all settings:
• Knowledge Base name
• IAM role
• S3 data source
• Embedding model
• Click “Create Knowledge Base.”

Step 3: Query and Test the Knowledge Base

With the Knowledge Base set up and linked to your SOP document, follow these steps to sync the data source and verify that the RAG pipeline works:

Step 1: Sync the Knowledge Base

1. Open the Amazon Bedrock Console.
2. Go to Knowledge bases in the left-hand menu.
3. Select your Knowledge Base (e.g., faq-chatbot-kb).
4. Navigate to Data Source, select your data source, and click Sync.
5. Wait for the status to change to Synced or Completed.

Note: If the data source is not synced, you won’t be able to test the Knowledge Base.

Step 2: Open the Test Interface

1. In the Knowledge Bases list, select your Knowledge Base.
2. Click Test Knowledge Base to open the interactive test panel.

Step 3: Run an In-Scope Query

1. In the test interface, configure Retrieval and Response Generation:

• Select Data Sources and Model.
• Choose Amazon → Titan Text Embeddings V2 as the model and click Apply.

2. Enter a question that exists in your SOP document, e.g., What does the Cloud Architect do in the architecture design phase?

3. Press Enter.

Expected Result: The system retrieves the correct text from the SOP document along with the file reference.

Step 4: Run an Out-of-Scope Query

1. Test a question not in your document, e.g., What is the cafeteria lunch menu?
2. Press Enter.

Expected Result: No answer is returned, confirming that the system retrieves information only from your data.

Step 5: Test Edge Cases and Paraphrased Queries

1. Test semantically similar questions, such as:

• Which team designs the infrastructure?
• What’s the role of the cloud architect?
• Who sets up the cloud architecture?

2. Press Enter for each query.

Expected Result: The system retrieves the correct response despite variations in wording, demonstrating semantic similarity matching.

Step 4: Build the Web Application

Follow these steps to build and deploy a simple web app that lets users ask questions and get answers from your Amazon Bedrock Knowledge Base.

Step 1: Create the Lambda Function (Backend)

1. Open the AWS Management Console → Lambda → Create function.
2. Choose:

• Author from scratch
• Function name: faq-kb-web-backend
• Runtime: Python 3.14
• Permissions: Create a new role with basic Lambda permissions.

3. Click the Create function.

Step 2: Add Bedrock Permissions to the Lambda Role

1. Go to IAM → Roles.
2. Find the role created for your Lambda function (name starts with faq-kb-web-backend).
3. Click the role → Add permissions → Attach policies → Search for Bedrock.
4. Attach the policy: AmazonBedrockFullAccess.

Step 3: Add Environment Variables

1. Go back to your Lambda function.
2. Navigate to Configuration → Environment variables → Edit.
3. Add these key-value pairs:

• BEDROCK_REGION = your Bedrock region (e.g., us-east-1)
• KNOWLEDGE_BASE_ID = your Knowledge Base ID (e.g., ABCD1234XYZ)

4. Save changes.

5. Under General configuration, increase the Timeout to 1 minute.

Step 4: Add Lambda Code

1. In the Code tab of your Lambda function, replace the default code with the provided Python script.

import json
import os
import boto3

# Environment variables:
# BEDROCK_REGION     -> e.g. "us-east-1"
# KNOWLEDGE_BASE_ID  -> your Bedrock Knowledge Base ID
# MODEL_ARN          -> ARN of the model you have access to
BEDROCK_REGION = os.environ.get("BEDROCK_REGION", "us-east-1")
KNOWLEDGE_BASE_ID = os.environ["KNOWLEDGE_BASE_ID"]
MODEL_ARN = os.environ["MODEL_ARN"]

bedrock_agent = boto3.client("bedrock-agent-runtime", region_name=BEDROCK_REGION)

def lambda_handler(event, context):
    """Handle HTTP requests from the browser via Lambda Function URL."""
    try:
        # Detect HTTP method (works for Function URL & API Gateway-style events)
        method = (
            event.get("requestContext", {})
            .get("http", {})
            .get("method")
            or event.get("httpMethod")
        )

        # 1) CORS preflight
        if method == "OPTIONS":
            return _response(200, {"message": "CORS preflight OK"})

        # 2) Only allow POST for normal requests
        if method != "POST":
            return _response(405, {"error": f"Method {method} not allowed"})

        # Parse JSON body from the browser
        body = json.loads(event.get("body") or "{}")

        question = (body.get("question") or "").strip()
        user_context = (body.get("context") or "").strip()

        if not question:
            return _response(400, {"error": "Field 'question' is required."})

        # Optional extra hint for retrieval
        full_input = question
        if user_context:
            full_input = f"{question}\\n\\nAdditional context from user:\\n{user_context}"

        # Call Bedrock Knowledge Base with explicit modelArn
        resp = bedrock_agent.retrieve_and_generate(
            input={"text": full_input},
            retrieveAndGenerateConfiguration={
                "type": "KNOWLEDGE_BASE",
                "knowledgeBaseConfiguration": {
                    "knowledgeBaseId": KNOWLEDGE_BASE_ID,
                    "modelArn": MODEL_ARN,
                },
            },
        )

        # ----- Extract answer text safely -----
        answer_text = ""
        output = resp.get("output", {})

        text_field = output.get("text")

        if isinstance(text_field, str):
            # Most common shape: text is a simple string
            answer_text = text_field.strip()
        elif isinstance(text_field, list):
            # Fallback if AWS ever returns a list of blocks
            for block in text_field:
                if isinstance(block, dict) and "text" in block:
                    answer_text += str(block["text"])
            answer_text = answer_text.strip()

        if not answer_text:
            answer_text = "No answer returned from Knowledge Base."

        # Optional: count retrieved references for display
        citation_count = 0
        for c in resp.get("citations", []):
            citation_count += len(c.get("retrievedReferences", []))

        result = {
            "answer": answer_text,
            "citation_count": citation_count,
        }

        return _response(200, result)

    except Exception as e:
        print("Error:", e)
        return _response(500, {"error": f"Internal error: {str(e)}"})

def _response(status_code, body):
    """Build HTTP response with CORS headers."""
    return {
        "statusCode": status_code,
        "headers": {
            "Content-Type": "application/json",
            "Access-Control-Allow-Origin": "*",
            "Access-Control-Allow-Headers": "Content-Type",
            "Access-Control-Allow-Methods": "OPTIONS,POST",
        },
        "body": json.dumps(body),
    }

2. Click Deploy.

Step 5: Enable the Lambda Function URL

1. In the Configuration section, go to Function URL → Create function URL.
2. Set Auth type to NONE → Click Save.
3. Copy the generated URL (e.g., https://abc123xyz.lambda-url.us-east-1.on.aws/)..)

Step 6: Create the Web Page

1. Create a new file on your computer named index.html.
2. Paste the provided HTML code into the file.

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8" />
  <title>FAQ Chatbot • Bedrock Knowledge Base</title>
  <meta name="viewport" content="width=device-width, initial-scale=1" />
  <style>
    body {
      font-family: system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
      background: #020617;
      color: #e5e7eb;
      margin: 0;
      padding: 24px 16px;
      display: flex;
      justify-content: center;
    }

    .container {
      width: 100%;
      max-width: 700px;
    }

    h1 {
      font-size: 24px;
      margin-bottom: 4px;
    }

    .subtitle {
      font-size: 14px;
      color: #9ca3af;
      margin-bottom: 16px;
    }

    .card {
      background: #020617;
      border: 1px solid #1f2937;
      border-radius: 14px;
      padding: 16px;
      margin-bottom: 16px;
      box-shadow: 0 10px 30px rgba(0, 0, 0, 0.5);
    }

    label {
      font-size: 14px;
      display: block;
      margin-bottom: 6px;
    }

    textarea {
      width: 100%;
      min-height: 80px;
      background: #030712;
      border-radius: 10px;
      border: 1px solid #1f2937;
      padding: 10px;
      color: #e5e7eb;
      font-size: 14px;
      resize: vertical;
      margin-bottom: 10px;
    }

    button {
      background: linear-gradient(135deg, #22d3ee, #3b82f6);
      color: #020617;
      border: none;
      border-radius: 999px;
      padding: 10px 20px;
      font-size: 14px;
      font-weight: 500;
      cursor: pointer;
    }

    button:disabled {
      opacity: 0.7;
      cursor: default;
    }

    .answer-title {
      font-size: 12px;
      text-transform: uppercase;
      color: #9ca3af;
      margin-bottom: 6px;
    }

    .answer-box {
      font-size: 14px;
      white-space: pre-wrap;
      max-height: 260px;
      overflow-y: auto;
    }

    .meta {
      margin-top: 8px;
      font-size: 12px;
      color: #9ca3af;
    }

    .error {
      margin-top: 8px;
      font-size: 13px;
      color: #f97373;
    }
  </style>
</head>
<body>
<div class="container">
  <h1>Ask questions about your knowledge base</h1>
  <p class="subtitle">
    Type a question, optionally add a hint, and let Amazon Bedrock answer using your Knowledge Base.
  </p>

  <div class="card">
    <label for="questionInput">Your question</label>
    <textarea id="questionInput"
              placeholder="e.g., What does the Cloud Architect own during the design phase?"></textarea>

    <label for="contextInput">Optional hint (for better retrieval)</label>
    <textarea id="contextInput"
              placeholder="Optional: add keywords, section names, or any extra clue..."></textarea>

    <button id="askBtn">Ask</button>
    <div id="error" class="error" style="display:none;"></div>
  </div>

  <div class="card">
    <div class="answer-title">Latest answer</div>
    <div id="answer" class="answer-box">No answer yet.</div>
    <div id="meta" class="meta"></div>
  </div>
</div>

<script>
  // TODO: Replace this with your actual Lambda Function URL
  const FUNCTION_URL = "YOUR_LAMBDA_FUNCTION_URL_HERE";

  const questionInput = document.getElementById("questionInput");
  const contextInput = document.getElementById("contextInput");
  const askBtn = document.getElementById("askBtn");
  const errorEl = document.getElementById("error");
  const answerEl = document.getElementById("answer");
  const metaEl = document.getElementById("meta");

  function showError(msg) {
    errorEl.style.display = "block";
    errorEl.textContent = msg;
  }

  function clearError() {
    errorEl.style.display = "none";
    errorEl.textContent = "";
  }

  askBtn.onclick = async () => {
    clearError();
    metaEl.textContent = "";

    const question = questionInput.value.trim();
    const context = contextInput.value.trim();

    if (!question) {
      showError("Please enter a question.");
      return;
    }

    askBtn.disabled = true;
    askBtn.textContent = "Asking...";

    try {
      const res = await fetch(FUNCTION_URL, {
        method: "POST",
        headers: { "Content-Type": "application/json" },
        body: JSON.stringify({ question, context })
      });

      if (!res.ok) {
        const text = await res.text();
        showError("Error from backend: " + text);
        answerEl.textContent = "No answer.";
        return;
      }

      const data = await res.json();

      if (data.error) {
        showError(data.error);
      }

      answerEl.textContent = data.answer || "No answer returned.";

      if (typeof data.citation_count === "number") {
        metaEl.textContent =
          "Answer is based on " + data.citation_count + " retrieved chunk(s) from your Knowledge Base.";
      }
    } catch (err) {
      console.error(err);
      showError("Network error calling Lambda Function URL. Check the browser console.");
      answerEl.textContent = "No answer.";
    } finally {
      askBtn.disabled = false;
      askBtn.textContent = "Ask";
    }
  };
</script>
</body>
</html>

3. Replace this line in the script: const FUNCTION_URL = “YOUR_LAMBDA_FUNCTION_URL_HERE”; with your Lambda Function URL (e.g., https://abc123xyz.lambda-url.us-east-1.on.aws/)..)

4. Save the file.

Step 7: Host the Web Page on S3

1. Go to Amazon S3 → Buckets → Create or open a bucket (e.g., faq-webapp-demo).
2. Click Upload → Add your index.html file.
3. In Permissions → Block public access, click Edit → Uncheck Block all public access → Save.
4. Add a bucket policy to allow public read access. Replace <your-bucket-name> with your bucket name:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowPublicReadForWebsite",
      "Effect": "Allow",
      "Principal": "*",
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::<your-bucket-name>/*"
    }
  ]
}

5. Save the changes and ensure index.html is publicly readable.

Step 8: Enable Static Website Hosting

1. In your bucket, go to Properties → Static website hosting → Edit
2. Choose Enable and set:

• Index document: index.html

3. Click Save.

4. Copy the Bucket website endpoint (e.g., http://<your-bucket-name>.s3-website-us-east-1.amazonaws.com).

Step 9: Test the Web App

1. Open the S3 website URL in your browser.
2. Type a question your Knowledge Base can answer (e.g., What does the Cloud Architect do during the design phase?).
3. Optionally, add a hint in the Optional hint box.
4. Click Ask.

Expected Results:

• An answer generated by Amazon Bedrock Knowledge Base, grounded in your indexed content.

Final Thoughts

This work not only gave me a wealth of technical knowledge — it gave me insight into how transformative AWS can be in tackling real-world problems. From the simplicity of serverless architecture to the power of RAG, every step reinforced how great AWS is as a platform for innovation.

And by the time I was done, I’d created more than just a chatbot. Taking on Big Challenges One Step at a Time. I felt I could handle whatever big problem I was facing. If you’re new to this or a wiz, this is a project for you. Why wait? Time to roll up your sleeves and build your own secure, enterprise-ready AI assistant today!

Chasing Dreams in the Cloud: Let’s Connect

Are you ready to start building your career on the clouds? Whether youʼre just getting started or already on your way, letʼs encourage and support each other to make our dreams in the cloud come true.

CloudComputing #TechProjects #AIArchitecture #AWSProjects #BuildingInTheCloud #CareerChange