Performing RAG with Knowledge Bases for Amazon Bedrock

Problem

Foundation models are trained on public data, which means they know about the world but nothing about your organization. When a customer service assistant gets asked about a specific product's warranty terms, internal return policy, or a historical support ticket, the model can only generate plausible-sounding text — it cannot retrieve accurate, company-specific facts it was never trained on. Building a custom RAG pipeline from scratch to solve this requires assembling an embedding model, a vector database, a chunking strategy, a retrieval layer, and a generation step — each with its own infrastructure overhead. For most teams, that complexity is prohibitive relative to the business value they need to unlock. The gap between a capable foundation model and a genuinely useful enterprise assistant comes down to one unsolved problem: grounding.

Solution

Amazon Bedrock Knowledge Bases provides a fully managed RAG pipeline that handles document ingestion, embedding generation, vector storage, semantic retrieval, and generation augmentation — without requiring any custom infrastructure. This project built a proof of concept for AnyCompany Consumer Electronics, tasked with answering customer service questions using internal documents: product descriptions, satisfaction surveys, support ticket histories, and sales records. Documents were uploaded to S3, ingested into a Knowledge Base backed by OpenSearch Serverless, and embedded using Amazon Titan Text Embeddings v2. The RetrieveAndGenerate API then combined semantic document retrieval with Nova Lite generation, producing answers that cited specific source documents rather than hallucinating generic responses. A direct comparison between a knowledge-base-augmented LLM response and a vanilla LLM response made the grounding improvement immediately concrete.

Skills Acquired

• Amazon Bedrock — the managed foundation model service used both as the generation backbone (Nova Lite) and as the RAG orchestration layer through its Knowledge Bases feature. Bedrock handles inference without requiring any model hosting infrastructure.
• Knowledge Bases — Amazon Bedrock's managed RAG feature. A Knowledge Base connects a document source to a vector store, runs the full embedding pipeline on ingestion, and exposes a single RetrieveAndGenerate API endpoint that handles retrieve-then-generate in one call.
• RAG (Retrieval Augmented Generation) — the architectural pattern at the core of this project. RAG retrieves relevant document chunks at query time and injects them into the model's context window, letting a frozen LLM answer questions about data it was never trained on — without retraining or fine-tuning.
• OpenSearch Serverless — the vector store that indexes document embeddings and serves approximate nearest-neighbor queries at retrieval time. Bedrock Knowledge Bases provisions and manages the OpenSearch collection automatically, requiring no direct OpenSearch configuration.
• Amazon S3 — the document source layer. Raw files — PDFs, CSVs, and text documents — were uploaded to an S3 bucket and synced into the Knowledge Base, which handled chunking and embedding on ingestion.
• Python — used to query the Knowledge Base programmatically via the RetrieveAndGenerate API, enabling scripted testing and integration into downstream application code.
• boto3 — the AWS Python SDK used to send retrieve-and-generate requests, inspect source attribution citations, and compare grounded versus ungrounded model responses in code.

What those tools accomplish together is best understood by seeing them in action — starting with the question that motivated the entire architecture.

Deep Dive

The hardest part of building a Retrieval Augmented Generation (RAG) system isn't the AI model — it's connecting proprietary data to an LLM in a way that's reliable, grounded, and maintainable. Most RAG tutorials show you how to wire up vector databases, embedding pipelines, and retrieval logic from scratch. Amazon Bedrock Knowledge Bases lets you skip that scaffolding entirely.

In this project, I played the role of an AI specialist at AnyCompany Consumer Electronics — a fictional company with a real-world problem. The VP of Customer Service wanted a chat assistant that could answer questions using the team's internal documents: product descriptions, customer satisfaction data, support ticket histories, and sales records. The goal was to build a proof of concept without building a custom RAG pipeline from scratch.

Why This Project?

My previous AWS project (Serverless AI Application) wired Lambda, API Gateway, and Amazon Bedrock together to invoke an LLM via API. But the LLM had no awareness of AnyCompany's internal data — it could only generate generic answers. RAG solves that problem by giving the model a memory of your organization's specific knowledge at query time.

Understanding how to implement RAG is increasingly central to applied AI work. Whether you're building enterprise chatbots, internal search tools, or domain-specific assistants, the ability to ground an LLM in proprietary data without retraining it is a foundational skill in the modern AI stack.

What this project covers:

• Creating an S3-backed data source and uploading structured + unstructured documents
• Creating and syncing an Amazon Bedrock Knowledge Base with a vector store (OpenSearch Serverless)
• Generating text embeddings using Amazon Titan Text Embeddings v2
• Testing RAG responses in the Bedrock console and analyzing source attribution
• Querying the knowledge base programmatically using the RetrieveAndGenerate API via Python and boto3

Architecture Overview

The knowledge base sits between the user's query and the foundation model, retrieving the most relevant document chunks and injecting them into the prompt as context.

How It Was Built

# Create bucket with randomized name
kbbucket=$(aws s3api create-bucket \
  --bucket kbbucket-$RANDOM-$RANDOM \
  --region us-east-1 \
  | cut -d '"' -f4 | cut -d '/' -f2 | sed -n 2p)
echo "bucket: "$kbbucket

# Upload all CSV and TXT files to the bucket
for csv in $(ls *.csv); do
  aws s3 cp $csv s3://$kbbucket/
done
for txt in $(ls *.txt); do
  aws s3 cp $txt s3://$kbbucket/
done

# Verify: 6 documents now in the bucket
aws s3 ls $kbbucket

import os
import boto3
import json
import sys

boto3_session = boto3.session.Session()
region = boto3_session.region_name

bedrock_agent_runtime_client = boto3.client(
    'bedrock-agent-runtime', region_name=region
)

# Knowledge Base ID (FMI-1) and foundation model ID (FMI-2)
kb_id = "JYOO2OEH0A"
print('kb_id = ' + kb_id)
model_id = "amazon.nova-lite-v1:0"
model_arn = f'arn:aws:bedrock:us-east-1::foundation-model/{model_id}'
sessionId = ""

def retrieveAndGenerate(input_value, kbId, model_arn, sessionId):
    """Query the knowledge base and generate a grounded response."""
    print(input_value, kbId, model_arn, sessionId)

    config = {
        "type": "KNOWLEDGE_BASE",
        "knowledgeBaseConfiguration": {
            "knowledgeBaseId": kbId,
            "modelArn": model_arn,
        },
    }

    if sessionId:
        print("sessionId is not empty")
        response = bedrock_agent_runtime_client.retrieve_and_generate(
            input={"text": input_value},
            retrieveAndGenerateConfiguration=config,
            sessionId=sessionId
        )
    else:
        print("sessionId is empty")
        response = bedrock_agent_runtime_client.retrieve_and_generate(
            input={"text": input_value},
            retrieveAndGenerateConfiguration=config,
        )

    print(json.dumps(response, indent=4, default=str))
    return response['sessionId']

# Accept prompt from command line
if len(sys.argv) < 2:
    print("Usage: python retrieve-generate.py '<your prompt>'")
    sys.exit(1)

input_value = sys.argv[1]
sessionId = retrieveAndGenerate(input_value, kb_id, model_arn, sessionId)

Key Takeaways

• RAG solves the knowledge cutoff problem without retraining. An LLM queried directly about AnyCompany's products or customers returns generic or hallucinated answers. With a knowledge base, those same queries return grounded, document-sourced responses.
• Embeddings are the bridge between text and semantic search. The Titan Text Embeddings v2 model converts both the documents and incoming queries into high-dimensional vectors. Proximity in vector space = semantic similarity — not keyword overlap.
• Source attribution is built in. Every response includes footnotes linking back to the specific document chunk — and by extension, the S3 object — that grounded the answer. This is critical for enterprise trust and auditability.
• Session IDs enable multi-turn conversation. The sessionId returned by the API links conversation turns, allowing follow-up questions like "Were most of those issues resolved?" to be answered with awareness of the prior question.
• Sync is required to ingest new data. The knowledge base isn't live — it reflects the state of the S3 bucket at the time of the last sync. Adding new documents requires re-syncing to regenerate embeddings and update the vector index.

Conclusion & Reflections

This project reinforced a core idea in applied AI: the model is often not the hard part. Getting the right data in front of the model — cleanly, reliably, with traceable sourcing — is where most of the engineering effort goes. Amazon Bedrock Knowledge Bases abstracts that plumbing, but understanding what it's doing underneath (chunking, embedding, vector search, prompt augmentation) makes you a better practitioner of RAG regardless of which tooling you use.

For AnyCompany's customer support team, the POC demonstrated that support staff could query their own historical data conversationally — finding out which customers prefer email, which product issues are most common, and whether prior tickets were resolved — without writing a single SQL query. That's a tangible productivity gain, and it's the kind of AI application that organizations are actively building.