Best Practices

AWS IAM Best Practices – Least Privilege & MFA

~ Last updated on : ~ jayendrapatil ~ 10 Comments

AWS IAM Best Practices

📋 Content Update Notice (June 2026)

This post has been significantly updated to align with the latest AWS IAM security best practices. Key changes include: use of AWS IAM Identity Center for centralized workforce access, mandatory MFA enforcement for all root users (completed June 2025), support for passkeys (FIDO2) as phishing-resistant MFA, IAM Roles Anywhere for external workloads, Resource Control Policies (RCPs) for resource-level guardrails, enhanced IAM Access Analyzer capabilities including unused access analysis and guided revocation, and centralized root access management in AWS Organizations.

AWS recommends the following AWS Identity and Access Management service – IAM Best Practices to secure AWS resources

Root Account – Protect & Minimize Root User Usage

  • Do not use the AWS Root account which has full access to all the AWS resources and services including the Billing information.
  • Permissions associated with the AWS Root account cannot be restricted.
  • Do not generate the access keys, if not required.
  • If already generated and not needed, delete the access keys.
  • If access keys are needed, rotate (change) the access key regularly.
  • Never share the Root account credentials or access keys, instead create IAM users or Roles to grant granular access.
  • Enable AWS multifactor authentication (MFA) on the AWS account — AWS now enforces MFA for all root users across all account types (as of June 2025).
  • Use phishing-resistant MFA methods such as passkeys or FIDO2 security keys for root account protection.
  • Use centralized root access management in AWS Organizations to monitor, remove, and prevent recovery of long-term root credentials across member accounts.
  • Root users must register MFA within 35 days of first sign-in attempt if not already enabled.

Use Federation with IAM Identity Center for Human Users

  • Don’t use the AWS root account credentials to access AWS, and don’t share the credentials with anyone else.
  • Use AWS IAM Identity Center (formerly AWS SSO) for centralized access management to provide workforce access to AWS accounts using temporary credentials.
  • Use an identity provider (IdP) for federated access to AWS accounts by assuming IAM roles, which provide temporary credentials.
  • IAM Identity Center supports integration with external identity providers (Okta, Azure AD, Google Workspace, etc.) or its own built-in identity store.
  • Create individual IAM users only for specific legacy use cases that cannot use federation (e.g., third-party tools, CodeCommit SSH access, Amazon Keyspaces).
  • For scenarios requiring IAM users with programmatic access, use long-term credentials only when temporary credentials via roles are not feasible.

Groups – Use groups to assign permissions to IAM users

  • Instead of defining permissions for individual IAM users, create groups and define the relevant permissions for each group as per the job function, and then associate IAM users to those groups.
  • Users in an IAM group inherit the permissions assigned to the group and a User can belong to multiple groups.
  • It is much easier to add new users, remove users and modify the permissions of a group of users.

Permission – Apply Least-Privilege Permissions

  • IAM user, by default, is created with no permissions.
  • Users should be granted LEAST PRIVILEGE as required to perform a task.
  • Starting with minimal permissions and adding to the permissions as required to perform the job function is far better than granting all access and trying to then tighten it down.
  • Get started with AWS managed policies for common use cases and move toward least-privilege with customer managed policies specific to your use cases.
  • Use IAM Access Analyzer policy generation to generate fine-grained policies based on actual access activity logged in CloudTrail.
  • Use IAM last accessed information to identify and remove unused permissions, policies, and credentials.

Passwords – Enforce strong password policy for users

  • Enforce users to create strong passwords and enforce them to rotate their passwords periodically.
  • Enable a strong password policy to define password requirements forcing users to create passwords with requirements like at least one capital letter, one number, and how frequently it should be rotated.

MFA – Require MFA for All Users

  • Enable MultiFactor Authentication (MFA) for all IAM users, especially those with privileged access to sensitive resources or APIs.
  • AWS now enforces MFA for root users across all account types — Organizations management accounts (May 2024), standalone accounts (June 2024), and all remaining root users (June 2025).
  • Use phishing-resistant MFA such as passkeys and FIDO2 security keys wherever possible.
  • Passkeys (supported since June 2024) use public key cryptography for strong, phishing-resistant authentication that works across devices.
  • AWS supports up to 8 MFA devices per root and IAM user.
  • Synced passkeys allow users to access their FIDO sign-in credentials across multiple devices without re-enrolling each device.
  • MFA prevents over 99% of password-related attacks.
  • If using IAM Identity Center, configure MFA capabilities within Identity Center when using its built-in identity store or AD Connector.

Role – Use Temporary Credentials with IAM Roles

  • Use roles for workloads instead of creating IAM users and hardcoding the credentials which can compromise the access and are also hard to rotate.
  • Roles have specific permissions and do not have a permanent set of credentials.
  • Roles provide a way to access AWS by relying on dynamically generated & automatically rotated temporary security credentials.
  • For workloads running on AWS compute services (EC2, Lambda, ECS), IAM roles deliver temporary credentials automatically.
  • For workloads running outside AWS, use:
    • IAM Roles Anywhere — obtain temporary credentials using X.509 certificates from your PKI (credentials valid up to 12 hours).
    • AssumeRoleWithSAML — using SAML assertions from an external IdP.
    • AssumeRoleWithWebIdentity — using JWT from an OIDC-compatible IdP.
    • AWS IoT Core — using Mutual TLS (MTLS) authentication for IoT devices.
  • Additional options for hybrid workloads: ECS Anywhere, EKS Hybrid Nodes, and Systems Manager Hybrid Activations all deliver temporary credentials to external compute resources.

Sharing – Delegate using roles

  • Allow users from same AWS account, another AWS account, or externally authenticated users (either through any corporate authentication service or through Google, Facebook etc) to use IAM roles to specify the permissions which can then be assumed by them.
  • A role can be defined that specifies what permissions the IAM users in the other account are allowed, and from which AWS accounts the IAM users are allowed to assume the role.

Rotation – Update Access Keys When Needed

  • Where possible, rely on temporary credentials instead of creating long-term credentials such as access keys.
  • For scenarios requiring IAM users with programmatic access and long-term credentials, update access keys when needed (e.g., when an employee leaves).
  • Use IAM access last used information to update and remove access keys safely.
  • Access keys allow creation of 2 active keys at the same time for a user. These can be used to rotate the keys.

Track & Review – Regularly Remove Unused Credentials

  • Remove IAM users, roles, permissions, policies, and credentials (passwords and access keys) that are not needed.
  • Use the IAM Credential report that lists all IAM users in the account and the status of their various credentials, including passwords, access keys, and MFA devices and usage patterns.
  • Use IAM last accessed information to identify users, roles, and permissions no longer needed.
  • Passwords and access keys that have not been used recently might be good candidates for removal.
  • Use IAM Access Analyzer unused access analysis to continuously identify unused roles, access keys, passwords, and unused service/action-level permissions across your organization.

Conditions – Use policy conditions for extra security

  • Define conditions under which IAM policies allow access to a resource.
  • Conditions would help provide finer access control to the AWS services and resources for e.g. access limited to a specific IP range, requiring TLS for all requests, allowing only encrypted requests for uploads to S3 buckets, or restricting access to specific AWS services like CloudFormation.

Auditing – Monitor activity in the AWS account

  • Enable logging features provided through CloudTrail, S3, CloudFront in AWS to determine the actions users have taken in the account and the resources that were used.
  • Log files show the time and date of actions, the source IP for an action, which actions failed due to inadequate permissions, and more.

Use IAM Access Analyzer

  • IAM Access Analyzer provides multiple capabilities to help achieve least privilege:
  • External access analysis — identifies resources shared with external principals and generates findings for public and cross-account access.
  • Unused access analysis — continuously identifies unused IAM roles, unused access keys, unused passwords, and unused service/action-level permissions across your organization (paid feature).
  • Policy generation — generates fine-grained least-privilege policies based on actual access activity captured in CloudTrail logs.
  • Policy validation — provides 100+ policy checks and actionable recommendations to ensure policies adhere to IAM best practices.
  • Custom policy checks — CheckNoNewAccess and CheckAccessNotGranted APIs to verify policies don’t grant unintended access.
  • Guided revocation — provides guidance to revoke unused permissions, with quick links to delete unused roles, access keys, and passwords.
  • Integrates with AWS Security Hub for centralized unused access findings.

Use Permissions Boundaries

  • Use IAM Permissions Boundaries to delegate permissions management within an account.
  • IAM permissions boundaries help set the maximum permissions that you delegate and that an identity-based policy can grant to an IAM role.
  • A permissions boundary does not grant permissions on its own.

Establish Permissions Guardrails Across Multiple Accounts

  • Use AWS Organizations to separate workloads using multiple accounts.
  • Use Service Control Policies (SCPs) to establish permissions guardrails controlling access for all IAM principals (roles and users) across accounts.
  • Use Resource Control Policies (RCPs) (launched November 2024) to establish permissions guardrails controlling access to AWS resources across your organization.
    • RCPs set the maximum available permissions on resources at the organization, OU, or account level.
    • RCPs complement SCPs — SCPs restrict what principals can do, RCPs restrict what can be done to resources.
    • RCPs help establish data perimeters by restricting external access to resources at scale.
  • SCPs and RCPs do not grant permissions — you must still attach identity-based or resource-based policies to grant access.

Use AWS Managed Policies and Move Toward Least Privilege

  • Start with AWS managed policies that grant permissions for common use cases and job functions.
  • AWS managed policies may not grant least-privilege for specific use cases as they are designed for all AWS customers.
  • Reduce permissions further by defining customer managed policies specific to your use cases.
  • Use AWS managed policies for job functions (e.g., ViewOnlyAccess, SystemAdministrator, DatabaseAdministrator) as starting points.

AWS Certification Exam Practice Questions

  • Questions are collected from Internet and the answers are marked as per my knowledge and understanding (which might differ with yours).
  • AWS services are updated everyday and both the answers and questions might be outdated soon, so research accordingly.
  • AWS exam questions are not updated to keep up the pace with AWS updates, so even if the underlying feature has changed the question might not be updated
  • Open to further feedback, discussion and correction.
  1. Your organization is preparing for a security assessment of your use of AWS. In preparation for this assessment, which two IAM best practices should you consider implementing? Choose 2 answers
    1. Create individual IAM users for everyone in your organization (May not be needed as can use IAM Identity Center with federation)
    2. Configure MFA on the root account and for privileged IAM users
    3. Assign IAM users and groups configured with policies granting least privilege access
    4. Ensure all users have been assigned and are frequently rotating a password, access ID/secret key, and X.509 certificate (Must be assigned only if using console or through command line)
  2. What are the recommended best practices for IAM? (Choose 3 answers)
    1. Grant least privilege
    2. Use the AWS account(root) for regular user
    3. Use Mutli-Factor Authentication (MFA)
    4. Store access key/private key in git
    5. Rotate credentials regularly
  3. Which of the below mentioned options is not a best practice to securely manage the AWS access credentials?
    1. Enable MFA for privileged users
    2. Create individual IAM users
    3. Keep rotating your secure access credentials at regular intervals
    4. Create strong access key and secret access key and attach to the root account
  4. Your CTO is very worried about the security of your AWS account. How best can you prevent hackers from completely hijacking your account?
    1. Use short but complex password on the root account and any administrators.
    2. Use AWS IAM Geo-Lock and disallow anyone from logging in except for in your city.
    3. Use MFA on all users and accounts, especially on the root account. (For increased security, it is recommend to configure MFA to help protect AWS resources. AWS now enforces MFA for all root users.)
    4. Don’t write down or remember the root account password after creating the AWS account.
  5. Fill the blanks: ____ helps us track AWS API calls and transitions, ____ helps to understand what resources we have now, and ____ allows auditing credentials and logins.
    1. AWS Config, CloudTrail, IAM Credential Reports
    2. CloudTrail, IAM Credential Reports, AWS Config
    3. CloudTrail, AWS Config, IAM Credential Reports
    4. AWS Config, IAM Credential Reports, CloudTrail
  6. A company wants to ensure that its workforce users access AWS accounts using temporary credentials and centralized access management. Which AWS service should they use? [Updated 2026]
    1. AWS Directory Service
    2. AWS IAM Identity Center
    3. Amazon Cognito
    4. AWS Certificate Manager
  7. An organization needs to provide temporary AWS credentials to workloads running on-premises that cannot use IAM Identity Center. Which service should they use? [Updated 2026]
    1. AWS STS AssumeRole
    2. IAM User access keys
    3. IAM Roles Anywhere
    4. AWS Secrets Manager
  8. Which IAM Access Analyzer capability helps identify unused IAM roles, access keys, and permissions across an AWS Organization? [Updated 2026]
    1. External access analysis
    2. Policy validation
    3. Unused access analysis
    4. Policy generation
  9. Which AWS Organizations policy type was introduced in November 2024 to centrally restrict access to AWS resources across an organization? [Updated 2026]
    1. Service Control Policies (SCPs)
    2. Tag Policies
    3. Backup Policies
    4. Resource Control Policies (RCPs)
  10. Which MFA method does AWS recommend as the most secure, phishing-resistant option for protecting AWS accounts? [Updated 2026]
    1. Virtual MFA device (authenticator app)
    2. SMS text message codes
    3. Hardware TOTP token
    4. Passkeys and FIDO2 security keys

References

Building Containers – Distroless, Buildpacks & AR

~ Last updated on : ~ jayendrapatil

Google Cloud Building Containers Best Practices

Package a single app per container

  • An “app” is considered to be a single piece of software, with a unique parent process, and potentially several child processes.
  • A container is designed to have the same lifecycle as the app it hosts, so each of the containers should contain only one app. When a container starts, so should the app, and when the app stops, so should the container. for e.g. in the case of the classic Apache/MySQL/PHP stack, each component must be hosted in a separate container.

Properly handle PID 1, signal handling, and zombie processes

  • Linux signals are the main way to control the lifecycle of processes inside a container.
  • The app within the container should handle the Linux signals, as well as the best practice of a single app per container should be implemented.
  • Process identifiers (PIDs) are unique identifiers that the Linux kernel gives to each process.
  • PIDs are namespace, i.e. the containers PIDs are different from the host and are mapped to the PIDs on the host system.
  • Docker and Kubernetes use signals to communicate with the processes inside containers, most notably to terminate them.
  • Both Docker and Kubernetes can only send signals to the process that has PID 1 inside a container.
  • For Signal handling and Zombie processes following can be followed
    • Run as PID 1 and register signal handlers
      • Launch the process with the CMD and/or ENTRYPOINT instructions in the Dockerfile, which would give the PID 1 to the process
      • Use the built-in exec command to launch the process from the shell script. The exec command replaces the script with the program and the process then inherits PID 1.
    • Enable process namespace sharing in Kubernetes
      • Process namespace sharing for a Pod can be enabled where Kubernetes uses a single process namespace for all the containers in that Pod.
      • Kubernetes Pod infrastructure container becomes PID 1 and automatically reaps orphaned processes.
    • Use a specialized init system
      • Init system such as tini created especially for containers that can be used to handle signals and reaps any zombie processes

Optimize for the Docker build cache

  • Images are built layer by layer, and in a Dockerfile, each instruction creates a layer in the resulting image.
  • Docker build cache can accelerate the building of container images.
  • During a build, when possible, Docker reuses a layer from a previous build and skips a potentially costly step.
  • Docker can use its build cache only if all previous build steps used it.

Use Multi-Stage Builds

  • Multi-stage builds allow separating the build environment from the runtime environment in a single Dockerfile using multiple FROM statements.
  • Build tools, compilers, and development dependencies are isolated in early stages and only the final artifacts are copied into the minimal runtime image.
  • This significantly reduces the final image size and attack surface.
  • Google Cloud Build supports multi-stage Dockerfiles natively and recommends separating building of the application from assembling its runtime container.
  • Example pattern: use a full SDK image to compile, then copy binaries into a distroless or minimal base image for production.

Use Distroless or Minimal Base Images

  • Distroless images (maintained by Google at gcr.io/distroless/) contain only the application runtime and its dependencies — no package managers, shells, or other programs.
  • Distroless images drastically reduce the attack surface and result in smaller image sizes with fewer CVE findings.
  • Available for multiple runtimes: Java, Python, Node.js, Go (static), .NET, and base/CC variants.
  • For cases where a shell is needed for debugging, consider using the :debug tag variants in non-production environments only.
  • Alternative minimal base images include Alpine Linux, Debian Slim, and Google’s managed base images.

Remove unnecessary tools

  • Remove unnecessary tools helps reduce the attack surface of the app by removing any unnecessary tools.
  • Avoid running as root inside the container: this method offers the first layer of security and could prevent attackers from modifying files
  • Launch the container in read-only mode using the --read-only flag from the docker run or by using the readOnlyRootFilesystem option in Kubernetes.
  • Apply seccomp profiles to containers to restrict system calls to the kernel — GKE supports the default containerd seccomp profile and custom profiles for additional hardening.
  • Use GKE Sandbox (gVisor) to run untrusted workloads with an additional kernel-level isolation layer that intercepts system calls.

Build the smallest image possible

  • Smaller image offers advantages such as faster upload and download times
  • To reduce the size of the image, install only what is strictly needed
  • Use multi-stage builds to separate build dependencies from runtime dependencies
  • Prefer distroless or scratch-based images for compiled languages (Go, Rust)
  • Combine RUN commands with && to reduce the number of layers and avoid leftover files from intermediate steps

Scan images for vulnerabilities

  • For vulnerabilities, as the containers are supposed to be immutable, the best practice is to rebuild the image, patches included, and redeploy it
  • As containers have a shorter lifecycle and a less well-defined identity than servers, a centralized inventory system would not work effectively
  • Artifact Analysis (formerly Container Analysis) can scan images for security vulnerabilities in publicly monitored packages, including OS, Java, Go, Python, and Node.js packages
  • Artifact Analysis provides both automatic scanning (triggered on every push to Artifact Registry) and on-demand scanning (manual scans of local or registry images)
  • Generate Software Bill of Materials (SBOM) for your container images to track all dependencies and license compliance
  • Integrate vulnerability scanning into CI/CD pipelines using Cloud Build and Binary Authorization to enforce that only verified, signed images are deployed

Using public image

  • Consider before using public images as you cannot control what’s inside them
  • Public image such as Debian or Alpine can be used as the base image and building everything on top of that image
  • Use Assured Open Source Software (Assured OSS) from Google for curated and tested open source packages
  • Pin images using SHA256 digest references (e.g., image@sha256:abc123...) rather than mutable tags like :latest to ensure reproducibility

Managed Base Images

  • Managed base images are base container images that are automatically patched by Google for security vulnerabilities, using the most recent patches available from the project upstream
  • Google provides Container-Optimized OS (cos) for running containers on Compute Engine and GKE nodes, which is automatically updated with security patches

Store Images in Artifact Registry

  • Container Registry was shut down on March 18, 2025 — all container images must now be stored in Artifact Registry
  • Artifact Registry supports both container images and non-container artifacts (Maven, npm, Python packages, etc.)
  • Artifact Registry provides integrated vulnerability scanning through Artifact Analysis, IAM-based access control, and regional/multi-regional storage options
  • Automatic migration tools are available to migrate existing gcr.io endpoints to Artifact Registry without downtime

Use Cloud Native Buildpacks

  • Google Cloud’s buildpacks transform application source code into production-ready OCI container images without requiring a Dockerfile
  • Buildpacks automatically detect the application language and framework, install dependencies, and produce optimized images following container best practices
  • Supported languages include Go, Java, Node.js, Python, .NET, Ruby, and PHP
  • Buildpacks produce reproducible builds and automatically apply security patches to the builder stack
  • Integrated with Cloud Build, Cloud Run, and App Engine for seamless source-to-production workflows
  • The default builder uses Ubuntu 24 as of 2025 (gcr.io/buildpacks/builder)

Supply Chain Security

  • Binary Authorization ensures that only trusted, signed container images are deployed to GKE, Cloud Run, or Distributed Cloud environments
  • Implement SLSA (Supply chain Levels for Software Artifacts) framework to establish provenance and verify the integrity of build artifacts
  • Software Delivery Shield provides an end-to-end software supply chain security solution across Google Cloud, covering source, build, artifacts, deployment, and runtime
  • Use attestations in CI/CD pipelines to cryptographically verify that images pass required checks (vulnerability scan, code review, testing) before deployment
  • Integrate with Security Command Center to view container vulnerability findings alongside other cloud security risks

GCP Certification Exam Practice Questions

  • Questions are collected from Internet and the answers are marked as per my knowledge and understanding (which might differ with yours).
  • GCP services are updated everyday and both the answers and questions might be outdated soon, so research accordingly.
  • GCP exam questions are not updated to keep up the pace with GCP updates, so even if the underlying feature has changed the question might not be updated
  • Open to further feedback, discussion and correction.
  1. When creating a secure container image, which two items should you incorporate into the build if possible?
    1. Use public container images as a base image for the app.
    2. Build the smallest image possible
    3. Use many container image layers to hide sensitive information.
    4. Package multiple applications in a container
  2. Your organization wants to ensure only signed and verified container images are deployed to your GKE clusters. Which Google Cloud service should you use?
    1. Artifact Analysis
    2. Cloud Armor
    3. Binary Authorization
    4. Container-Optimized OS
  3. Which Google Cloud service should you use to store and manage your container images now that Container Registry has been shut down?
    1. Cloud Storage
    2. Artifact Registry
    3. Container Registry (deprecated)
    4. Cloud Build
  4. You want to containerize your application without writing a Dockerfile. Which Google Cloud feature allows source-to-image transformation?
    1. Cloud Build triggers
    2. Cloud Native Buildpacks
    3. Container-Optimized OS
    4. Artifact Analysis
  5. Which type of base image is recommended by Google for production containers to minimize the attack surface?
    1. Ubuntu full image
    2. Alpine with development tools
    3. Distroless images
    4. CentOS base image
  6. Your team needs to automatically scan container images for OS and language package vulnerabilities every time an image is pushed to the registry. Which Google Cloud feature provides this?
    1. Cloud Build steps
    2. Binary Authorization attestations
    3. Artifact Analysis automatic scanning
    4. GKE Security Posture dashboard

References

Architecting for the Cloud – AWS Best Practices – Whitepaper – Certification

~ Last updated on : ~ jayendrapatil ~ 5 Comments

Architecting for the Cloud – AWS Best Practices

📋 Important Note: Whitepaper Superseded

The original “Architecting for the Cloud: AWS Best Practices” whitepaper (last updated October 2018) has been superseded by the AWS Well-Architected Framework.

The Well-Architected Framework is now organized into six pillars: Operational Excellence, Security, Reliability, Performance Efficiency, Cost Optimization, and Sustainability (added in 2021). It receives continuous updates — most recently in November 2024 and April 2025.

This post is maintained for certification study purposes as the core architectural principles remain relevant.

Architecting for the Cloud – AWS Best Practices whitepaper provides architectural patterns and advice on how to design systems that are secure, reliable, high performing, and cost efficient

AWS Design Principles

Scalability

  • While AWS provides virtually unlimited on-demand capacity, the architecture should be designed to take advantage of those resources
  • There are two ways to scale an IT architecture
    • Vertical Scaling
      • takes place through increasing specifications of an individual resource for e.g. updating EC2 instance type with increasing RAM, CPU, IOPS, or networking capabilities
      • will eventually hit a limit, and is not always a cost effective or highly available approach
      • AWS Graviton-based instances (Graviton4 as of 2024) offer up to 40% better price-performance, making vertical scaling more cost-effective
    • Horizontal Scaling
      • takes place through increasing number of resources for e.g. adding more EC2 instances or EBS volumes
      • can help leverage the elasticity of cloud computing
      • not all the architectures can be designed to distribute their workload to multiple resources
      • applications designed should be stateless,
        • that needs no knowledge of previous interactions and stores no session information
        • capacity can be increased and decreased, after running tasks have been drained
      • State, if needed, can be implemented using
        • Low latency external store, for e.g. DynamoDB, ElastiCache (Redis or Memcached), to maintain state information
        • Session affinity, for e.g. ELB sticky sessions, to bind all the transactions of a session to a specific compute resource. However, it cannot be guaranteed or take advantage of newly added resources for existing sessions
      • Load can be distributed across multiple resources using
        • Push model, for e.g. through ELB where it distributes the load across multiple EC2 instances
        • Pull model, for e.g. through SQS or Kinesis where multiple consumers subscribe and consume
      • Distributed processing, for e.g. using EMR or Kinesis, helps process large amounts of data by dividing task and its data into many small fragments of works

Disposable Resources Instead of Fixed Servers

  • Resources need to be treated as temporary disposable resources rather than fixed permanent on-premises resources before
  • AWS focuses on the concept of Immutable infrastructure
    • servers once launched, is never updated throughout its lifetime.
    • updates can be performed on a new server with latest configurations,
    • this ensures resources are always in a consistent (and tested) state and easier rollbacks
  • AWS provides multiple ways to instantiate compute resources in an automated and repeatable way
    • Bootstrapping
      • scripts to configure and setup for e.g. using EC2 user data scripts and cloud-init to install software or copy resources and code
    • Golden Images
      • a snapshot of a particular state of that resource,
      • faster start times and removes dependencies to configuration services or third-party repositories
      • EC2 Image Builder can automate creation, testing, and distribution of golden AMIs
    • Containers
      • AWS supports container workloads through Amazon ECS, Amazon EKS, and AWS Fargate (serverless containers)
      • Docker allows packaging a piece of software in a Docker Image, which is a standardized unit for software development, containing everything the software needs to run: code, runtime, system tools, system libraries, etc
      • AWS Fargate provides serverless compute for containers, eliminating the need to manage underlying EC2 instances
  • Infrastructure as Code
    • AWS assets are programmable, techniques, practices, and tools from software development can be applied to make the whole infrastructure reusable, maintainable, extensible, and testable.
    • AWS provides services for IaC deployment:
      • AWS CloudFormation – declarative JSON/YAML templates for provisioning AWS resources
      • AWS CDK (Cloud Development Kit) – define infrastructure using familiar programming languages (TypeScript, Python, Java, Go, C#) that synthesize to CloudFormation
      • AWS SAM (Serverless Application Model) – simplified CloudFormation for serverless applications
    • Note: AWS OpsWorks reached End of Life on May 26, 2024 and is no longer available. Use AWS Systems Manager, CloudFormation, or CDK as alternatives.

Automation

  • AWS provides various automation tools and services which help improve system’s stability, efficiency and time to market.
    • Elastic Beanstalk
      • a PaaS that allows quick application deployment while handling resource provisioning, load balancing, auto scaling, monitoring etc
    • EC2 Auto Recovery
      • creates CloudWatch alarm that monitors an EC2 instance and automatically recovers it if it becomes impaired.
      • A recovered instance is identical to the original instance, including the instance ID, private & Elastic IP addresses, and all instance metadata.
      • Instance is migrated through reboot, in memory contents are lost.
    • Auto Scaling
      • allows maintain application availability and scale the capacity up or down automatically as per defined conditions
      • supports predictive scaling that uses machine learning to forecast traffic and proactively scale capacity
    • CloudWatch Alarms
      • allows SNS triggers to be configured when a particular metric goes beyond a specified threshold for a specified number of periods
    • Amazon EventBridge (formerly CloudWatch Events)
      • allows real-time stream of system events that describe changes in AWS resources
      • extends capabilities with partner event sources, Schema Registry, and EventBridge Pipes for point-to-point integrations
      • EventBridge Scheduler supports one-time and recurring schedules with built-in retry policies
    • AWS Systems Manager
      • provides operational management for AWS resources including patch management, configuration compliance, and automated runbooks
      • replaces the need for OpsWorks with features like State Manager, Automation, and Run Command
    • Lambda Scheduled Events
      • allows Lambda function creation and direct AWS Lambda to execute it on a regular schedule via EventBridge Scheduler.

Loose Coupling

  • AWS helps loose coupled architecture that reduces interdependencies, a change or failure in a component does not cascade to other components
    • Asynchronous Integration
      • does not involve direct point-to-point interaction but usually through an intermediate durable storage layer for e.g. SQS, Kinesis, EventBridge
      • decouples the components and introduces additional resiliency
      • suitable for any interaction that doesn’t need an immediate response and an ack that a request has been registered will suffice
    • Service Discovery
      • allows new resources to be launched or terminated at any point in time and discovered as well for e.g. using ELB as a single point of contact with hiding the underlying instance details or Route 53 zones to abstract load balancer’s endpoint
      • AWS Cloud Map provides service discovery for cloud resources, allowing applications to discover services via API calls, DNS queries, or directly through the SDK
    • Well-Defined Interfaces
      • allows various components to interact with each other through specific, technology agnostic interfaces for e.g. RESTful APIs with API Gateway
      • Amazon API Gateway supports REST APIs, HTTP APIs, and WebSocket APIs for real-time communication

Services, Not Servers

  • AWS encourages leveraging managed services and serverless architectures to reduce operational overhead
    • Serverless compute – AWS Lambda for event-driven functions, AWS Fargate for serverless containers
    • Managed databases – Amazon RDS, DynamoDB, Aurora Serverless for auto-scaling relational databases
    • Application integration – SQS, SNS, EventBridge, Step Functions for workflow orchestration
    • API management – API Gateway for creating, publishing, and managing APIs at scale

Databases

  • AWS provides different categories of database technologies
    • Relational Databases (RDS)
      • normalizes data into well-defined tabular structures known as tables, which consist of rows and columns
      • provide a powerful query language, flexible indexing capabilities, strong integrity controls, and the ability to combine data from multiple tables in a fast and efficient manner
      • allows vertical scalability by increasing resources and horizontal scalability using Read Replicas for read capacity and sharding or data partitioning for write capacity
      • provides High Availability using Multi-AZ deployment, where data is synchronously replicated
      • Amazon Aurora provides MySQL and PostgreSQL-compatible databases with up to 5x and 3x better throughput respectively, with automatic storage scaling up to 128 TB
      • Aurora Serverless v2 scales capacity automatically based on application demand, ideal for variable workloads
    • NoSQL Databases (DynamoDB)
      • provides databases that trade some of the query and transaction capabilities of relational databases for a more flexible data model that seamlessly scales horizontally
      • perform data partitioning and replication to scale both the reads and writes in a horizontal fashion
      • DynamoDB service synchronously replicates data across three facilities in an AWS region to provide fault tolerance in the event of a server failure or Availability Zone disruption
      • DynamoDB Global Tables provide multi-region, active-active replication for globally distributed applications
      • DynamoDB On-Demand mode eliminates capacity planning by automatically scaling to accommodate workloads
    • Data Warehouse (Redshift)
      • Specialized type of relational database, optimized for analysis and reporting of large amounts of data
      • Redshift achieves efficient storage and optimum query performance through a combination of massively parallel processing (MPP), columnar data storage, and targeted data compression encoding schemes
      • Redshift MPP architecture enables increasing performance by increasing the number of nodes in the data warehouse cluster
      • Redshift Serverless automatically provisions and scales capacity, allowing analytics without cluster management
    • Purpose-Built Databases
      • Amazon ElastiCache – in-memory caching (Redis, Memcached) for sub-millisecond latency
      • Amazon Neptune – graph database for highly connected datasets
      • Amazon Timestream – time series database for IoT and operational applications
      • Amazon MemoryDB for Redis – Redis-compatible, durable, in-memory database
  • For more details refer to AWS Storage Options Whitepaper

Removing Single Points of Failure

  • AWS provides ways to implement redundancy, automate recovery and reduce disruption at every layer of the architecture
  • AWS supports redundancy in the following ways
    • Standby Redundancy
      • When a resource fails, functionality is recovered on a secondary resource using a process called failover.
      • Failover will typically require some time before it completes, and during that period the resource remains unavailable.
      • Secondary resource can either be launched automatically only when needed (to reduce cost), or it can be already running idle (to accelerate failover and minimize disruption).
      • Standby redundancy is often used for stateful components such as relational databases.
    • Active Redundancy
      • requests are distributed to multiple redundant compute resources, if one fails, the rest can simply absorb a larger share of the workload.
      • Compared to standby redundancy, it can achieve better utilization and affect a smaller population when there is a failure.
  • AWS supports replication
    • Synchronous replication
      • acknowledges a transaction after it has been durably stored in both the primary location and its replicas.
      • protects data integrity from the event of a primary node failure
      • used to scale read capacity for queries that require the most up-to-date data (strong consistency).
      • compromises performance and availability
    • Asynchronous replication
      • decouples the primary node from its replicas at the expense of introducing replication lag
      • used to horizontally scale the system’s read capacity for queries that can tolerate that replication lag.
    • Quorum-based replication
      • combines synchronous and asynchronous replication to overcome the challenges of large-scale distributed database systems
      • Replication to multiple nodes can be managed by defining a minimum number of nodes that must participate in a successful write operation
  • AWS provide services to reduce or remove single point of failure
    • Regions, Availability Zones with multiple data centers
    • ELB or Route 53 to configure health checks and mask failure by routing traffic to healthy endpoints
    • Auto Scaling to automatically replace unhealthy nodes
    • EC2 auto-recovery to recover unhealthy impaired nodes
    • S3, DynamoDB with data redundantly stored across multiple facilities
    • Multi-AZ RDS, Aurora (6 copies across 3 AZs), and Read Replicas
    • ElastiCache Redis engine supports replication with automatic failover
    • AWS Elastic Disaster Recovery (DRS) for continuous replication and automated recovery of on-premises and cloud-based applications
  • For more details refer to AWS Disaster Recovery Whitepaper

Optimize for Cost

  • AWS can help organizations reduce capital expenses and drive savings as a result of the AWS economies of scale
  • AWS provides different options which should be utilized as per use case –
    • EC2 pricing models:
      • On-Demand – pay per second/hour with no commitment
      • Savings Plans – commit to a consistent amount of usage (measured in $/hr) for 1 or 3 years; Compute Savings Plans (up to 66% savings) and EC2 Instance Savings Plans (up to 72% savings)
      • Reserved Instances – capacity reservation with up to 72% discount for 1 or 3 year terms
      • Spot Instances – up to 90% discount for fault-tolerant, flexible workloads using spare capacity
      • Dedicated Hosts – single-tenant hardware for compliance and BYOL licensing
    • AWS Graviton instances for up to 40% better price-performance over comparable x86 instances
    • AWS Cost Optimization Hub, Trusted Advisor, and AWS Compute Optimizer to identify cost savings opportunities
    • S3 storage classes:
      • S3 Standard – frequently accessed data
      • S3 Intelligent-Tiering – automatic cost optimization for data with unknown or changing access patterns
      • S3 Standard-Infrequent Access (S3 Standard-IA) – infrequently accessed data
      • S3 One Zone-IA – infrequently accessed data not requiring multi-AZ resilience
      • S3 Glacier Instant Retrieval, Flexible Retrieval, and Deep Archive – long-term archive storage
      • S3 Express One Zone – single-digit millisecond latency for most frequently accessed data (up to 10x faster than S3 Standard)
    • EBS volumes – General Purpose SSD (gp3), Provisioned IOPS SSD (io2 Block Express), Throughput Optimized HDD (st1), Cold HDD (sc1). Note: Magnetic (standard) is a previous-generation volume type; gp3 is recommended as default.
    • Cost Allocation tags to identify costs based on tags
    • Auto Scaling to horizontally scale the capacity up or down based on demand
    • Lambda and Fargate based serverless architectures to never pay for idle or redundant resources
    • Utilize managed services where scaling is handled by AWS for e.g. ELB, CloudFront, Kinesis, SQS, Amazon OpenSearch Service etc.

Caching

  • Caching improves application performance and increases the cost efficiency of an implementation
    • Application Data Caching
      • provides services that help store and retrieve information from fast, managed, in-memory caches
      • Amazon ElastiCache is a web service that makes it easy to deploy, operate, and scale an in-memory cache in the cloud and supports two open-source in-memory caching engines: Memcached and Redis
      • Amazon DynamoDB Accelerator (DAX) provides a fully managed, in-memory cache for DynamoDB with microsecond response times
    • Edge Caching
      • allows content to be served by infrastructure that is closer to viewers, lowering latency and giving high, sustained data transfer rates needed to deliver large popular objects to end users at scale.
      • Amazon CloudFront is Content Delivery Network (CDN) consisting of 600+ Points of Presence (edge locations and regional caches), that allows copies of static and dynamic content to be cached
      • CloudFront Functions and Lambda@Edge enable running code at edge locations for request/response manipulation

Security

  • AWS works on shared security responsibility model
    • AWS is responsible for the security of the underlying cloud infrastructure
    • you are responsible for securing the workloads you deploy in AWS
  • AWS also provides ample security features
    • IAM to define a granular set of policies and assign them to users, groups, and AWS resources
    • IAM roles to assign short term credentials to resources, which are automatically distributed and rotated
    • AWS IAM Identity Center (formerly AWS SSO) for centralized workforce identity management and single sign-on across AWS accounts and applications
    • Amazon Cognito, for mobile and web applications, which allows client devices to get controlled access to AWS resources via temporary tokens
    • VPC to isolate parts of infrastructure through the use of subnets, security groups, and routing controls
    • AWS WAF to help protect web applications from SQL injection, cross-site scripting, and other common exploits with managed rule groups
    • CloudWatch logs to collect logs centrally as the servers are temporary
    • CloudTrail for auditing AWS API calls, which delivers a log file to S3 bucket. Logs can then be stored in an immutable manner and automatically processed to either notify or even take action on your behalf, protecting your organization from non-compliance
    • AWS Security Hub – unified security posture management that aggregates findings from GuardDuty, Inspector, Macie, and partner tools with automated compliance checks
    • Amazon GuardDuty – intelligent threat detection using machine learning, anomaly detection, and integrated threat intelligence to identify malicious activity
    • Amazon Inspector – automated vulnerability management that continuously scans EC2 instances, container images (ECR), Lambda functions, and code repositories for software vulnerabilities
    • AWS Config for continuous compliance monitoring, and AWS Trusted Advisor for best practice recommendations across cost, performance, security, fault tolerance, and service limits
  • For more details refer to AWS Security Whitepaper

AWS Well-Architected Framework

  • The AWS Well-Architected Framework is the successor to this whitepaper and provides comprehensive guidance for building secure, high-performing, resilient, and efficient infrastructure
  • The Framework is built on six pillars:
    • Operational Excellence – running and monitoring systems to deliver business value and continually improve processes and procedures
    • Security – protecting information and systems through risk assessments, mitigation strategies, and security best practices
    • Reliability – ensuring workloads perform correctly and consistently, with ability to recover from failures and meet demand
    • Performance Efficiency – using computing resources efficiently to meet requirements and maintain efficiency as demand changes
    • Cost Optimization – avoiding unnecessary costs through understanding spending, selecting the right resources, and scaling to meet needs without overspending
    • Sustainability (added 2021) – minimizing environmental impacts by reducing energy consumption and increasing efficiency of cloud workloads
  • The AWS Well-Architected Tool in the AWS Management Console allows workload reviews against framework best practices
  • AWS also provides Well-Architected Lenses for specific workload types (Serverless, SaaS, Machine Learning, Data Analytics, IoT, etc.)

References