GenAI Application Architecture on AWS – Bedrock, RAG, Agents & Guardrails

GenAI Application Architecture on AWS — Overview

This reference architecture shows a production-ready Generative AI application built on Amazon Bedrock, combining RAG for knowledge grounding, Agents for task execution, Guardrails for safety, and CloudWatch for observability. This is the standard pattern tested across AIF-C01, AIP-C01, and SAP-C02 certification exams.

Architecture Components — Deep Dive

1. User & Application Layer

2. Bedrock Guardrails (Input & Output)

Guardrails evaluate both the user input AND the model output independently:

• Input side: Blocks prompt injection attacks, inappropriate content, and denied topics before reaching the FM
• Output side: Filters harmful content, masks PII, and verifies contextual grounding after the FM responds
• Key insight: Guardrails are independent of the model — they enforce rules even if prompt engineering fails

3. Bedrock Agent (Orchestration)

The Agent is the “brain” that decomposes complex user requests into actionable steps:

• Orchestration strategies: Default (ReAct), Custom (Lambda-based), or Optimized for specific models
• Action Groups: Lambda functions or API schemas (OpenAPI) that the Agent can invoke as tools
• Memory: Session memory (within conversation) or summary memory (across conversations)
• Return of Control: Pause execution and ask the application for human approval before sensitive actions
• Code Interpreter: Execute Python code for data analysis, calculations, chart generation

4. Knowledge Base (RAG Pipeline)

Provides grounded, factual responses from your enterprise data:

• Ingestion: Documents → Chunking → Embedding → Vector Store (runs on sync schedule)
• Query: User query → Embed → Vector Search → Top-K retrieval → Reranking
• Generation: Retrieved context + User query → FM → Grounded response with citations
• Grounding check: Guardrails contextual grounding verifies response faithfulness to sources

5. Foundation Model

6. Observability & Evaluation

• CloudWatch Metrics: InvocationCount, InvocationLatency, ThrottlingCount, InputTokenCount, OutputTokenCount
• Model Invocation Logging: Full prompt + response logged to S3 or CloudWatch Logs (opt-in, for audit/debugging)
• CloudTrail: API-level audit — who invoked which model, when, from which IP
• LLM-as-Judge: Bedrock Model Evaluation uses a “judge” FM to score responses on accuracy, relevance, toxicity, and faithfulness
• Human evaluation: Route a sample of responses to human reviewers via SageMaker Ground Truth

Data Flow — Request Lifecycle

1. User sends request → API Gateway authenticates (Cognito/IAM) and rate-limits
2. Lambda/ECS receives → Constructs session context, calls Bedrock Agent
3. Guardrails INPUT check → Evaluates user message for prompt attacks, harmful content, denied topics
4. Agent orchestrates → Decides: does it need tools? Knowledge Base? Code execution?
5. Knowledge Base retrieval → Embeds query → Vector search → Returns top-K chunks with metadata
6. Action Group execution → Calls Lambda functions for external APIs, databases, calculations
7. FM generation → Foundation model receives (system prompt + retrieved context + user query + tool results) → Generates response
8. Guardrails OUTPUT check → Evaluates response for content safety, PII, grounding faithfulness
9. Response returned → With citations, session updated, metrics emitted to CloudWatch

RAG vs Fine-tuning — When to Use Which

RAG and fine-tuning are complementary customization techniques. RAG gives the model new knowledge (by injecting retrieved documents as context). Fine-tuning gives the model new behavior (by modifying its weights through training). They solve different problems.

Decision Guide

• “Answer questions from our internal docs” → RAG
• “Respond in our brand voice with specific formatting” → Fine-tuning
• “Keep answers current as policies change weekly” → RAG
• “Cite the exact source paragraph for compliance” → RAG
• “Brand voice + factual grounding from docs” → Fine-tune + RAG together

Best practice: Start with RAG (no training, immediate results). Add fine-tuning only when prompt engineering + RAG can’t achieve the desired output behavior.

Design Decisions & Trade-offs

Scaling Considerations

• Throughput: Bedrock has per-model, per-region quotas (tokens/min). Request quota increases early. Use cross-region inference for automatic overflow.
• Latency: First-token latency depends on model size. Use streaming (InvokeModelWithResponseStream) for perceived speed. Nova Micro/Haiku for lowest latency.
• Cost optimization: Batch Inference (50% discount) for offline processing. Provisioned Throughput for predictable high-volume. Cache frequent queries in ElastiCache.
• Concurrency: Lambda concurrency limits may throttle. Use reserved concurrency + SQS buffering for spikes.

Security Architecture

• Network: VPC endpoints (PrivateLink) for Bedrock — no internet traversal
• Encryption: Data encrypted at rest (KMS) and in transit (TLS 1.2+). Customer-managed keys for Knowledge Base data.
• IAM: Least-privilege policies per service. Agent execution role scoped to specific action groups.
• Data isolation: Your data never leaves your account. Not used for model training (Bedrock guarantee).
• Guardrails: Defense-in-depth — even if prompt injection bypasses system prompt, Guardrails block harmful output.

Exam Tips by Certification

AWS Certification Exam Practice Questions

Question 1:

A company deploys a GenAI chatbot using Bedrock Agents with Knowledge Bases. Users report that the Agent sometimes executes sensitive actions (refund processing) without confirmation. How should the architect prevent unauthorized automated actions?

1. Add Guardrails with denied topics for refunds
2. Configure Return of Control for the refund action group
3. Remove the refund Lambda from the Agent’s action groups
4. Set the Agent’s orchestration to custom with a confirmation step

Question 2:

A financial services company’s RAG system occasionally includes information from documents that the requesting user shouldn’t have access to. The Knowledge Base contains documents from multiple departments with different access levels. What is the MOST scalable solution?

1. Create separate Knowledge Bases per department
2. Use metadata filtering with user-department tags at query time
3. Implement row-level security on the vector store
4. Add Guardrails to filter department-specific content from responses

Question 3:

An enterprise wants to monitor their Bedrock application for quality degradation over time. They need to automatically detect when the model starts generating lower-quality responses compared to their baseline. Which approach is MOST appropriate?

1. CloudWatch anomaly detection on InvocationLatency metric
2. Bedrock Model Evaluation with LLM-as-Judge on a scheduled basis
3. Guardrails contextual grounding score tracked over time
4. Manual review of CloudWatch Logs weekly

Question 4:

A startup’s Bedrock chatbot experiences throttling during peak hours (5x normal traffic). They need to handle traffic spikes without pre-provisioning capacity. Which combination addresses this with MINIMAL operational overhead?

1. Provisioned Throughput with auto-scaling
2. Cross-region inference profile + SQS queue for overflow
3. Multiple model IDs with application-level load balancing
4. Switch to SageMaker endpoints with auto-scaling policies

Question 5:

An architect needs to ensure their Bedrock application’s prompts and responses never traverse the public internet for compliance requirements. Which configuration achieves this?

1. Enable encryption with customer-managed KMS keys
2. Configure VPC endpoints (PrivateLink) for Bedrock and deploy application in a private subnet
3. Use Bedrock in a dedicated tenancy mode
4. Enable model invocation logging to a VPC-bound S3 bucket

Frequently Asked Questions

How does the complete GenAI architecture work end-to-end?

User request enters through API Gateway, passes to Lambda/ECS which calls a Bedrock Agent. The Agent’s request is first checked by Guardrails (input). The Agent decides whether to use Knowledge Bases (RAG), Action Groups (tools), or Code Interpreter. The Foundation Model generates a response using the assembled context, which is then checked by Guardrails (output) before returning to the user with citations.

When should I use an Agent vs direct Knowledge Base query?

Use direct Knowledge Base (RetrieveAndGenerate) for simple document Q&A. Use an Agent when the task requires multiple steps, tool use (calling APIs, querying databases), conditional logic, or combining RAG with actions. Agents add latency but handle complex workflows.

How do I monitor a Bedrock application in production?

Enable CloudWatch metrics for invocation count/latency/errors, model invocation logging for full prompt/response audit trails, CloudTrail for API access auditing, and schedule Bedrock Model Evaluation with LLM-as-Judge to track quality over time. Set CloudWatch alarms on error rates and latency percentiles.