AWS CloudFormation Best Practices – Certification

AWS CloudFormation Best Practices

  • AWS CloudFormation Best Practices are based on real-world experience from current AWS CloudFormation customers
  • AWS CloudFormation Best Practices help provide guidelines on
    • how to plan and organize stacks,
    • create templates that describe resources and the software applications that run on them,
    • and manage stacks and their resources

Required Mainly for Developer, SysOps Associate & DevOps Professional Exam

Planning and Organizing

Organize Your Stacks By Lifecycle and Ownership

  • Use the lifecycle and ownership of the AWS resources to help you decide what resources should go in each stack.
  • By grouping resources with common lifecycles and ownership, owners can make changes to their set of resources by using their own process and schedule without affecting other resources.
  • For e.g. Consider an Application using Web and Database instances. Both the Web and Database have a different lifecycle and usually the ownership lies with different teams. Maintaining both in a single stack would need communication and co-ordination between different teams introducing complexity. It would be best to have different stacks owned by the respective teams, so that they can update their resources without impacting each others’s stack.

Use Cross-Stack References to Export Shared Resources

  • With multiple stacks, there is usually a need to refer values and resources across stacks.
  • Use cross-stack references to export resources from a stack so that other stacks can use them
  • Stacks can use the exported resources by calling them using the Fn::ImportValue function.
  • For e.g. Web stack would always need resources from the Network stack like VPC, Subnets etc.

Use IAM to Control Access

  • Use IAM to control access to
    • what AWS CloudFormation actions users can perform, such as viewing stack templates, creating stacks, or deleting stacks
    • what actions CloudFormation can perform on resources on their behalf
  • Remember, having access to CloudFormation does not provide user with access to AWS resources. That needs to be provided separately.
  • To separate permissions between a user and the AWS CloudFormation service, use a service role. AWS CloudFormation uses the service role’s policy to make calls instead of the user’s policy.

Verify Quotas for All Resource Types

  • Ensure that stack can create all the required resources without hitting the AWS account limits.

Reuse Templates to Replicate Stacks in Multiple Environments

  • Reuse templates to replicate infrastructure in multiple environments
  • Use parameters, mappings, and conditions sections to customize and make templates reusable
  • for e.g. creating the same stack in development, staging and production environment with different instance types, instance counts etc.

Use Nested Stacks to Reuse Common Template Patterns

  • Nested stacks are stacks that create other stacks.
  • Nested stacks separate out the common patterns and components to create dedicated templates for them, preventing copy pasting across stacks.
  • for e.g. a standard load balancer configuration can be created as nested stack and just used by other stacks

Creating templates

Do Not Embed Credentials in Your Templates

  • Use input parameters to pass in sensitive information such as DB password whenever you create or update a stack.
  • Use the NoEcho property to obfuscate the parameter value.

Use AWS-Specific Parameter Types

  • For existing AWS-specific values, such as existing Virtual Private Cloud IDs or an EC2 key pair name, use AWS-specific parameter types
  • AWS CloudFormation can quickly validate values for AWS-specific parameter types before creating your stack.

Use Parameter Constraints

  • Use Parameter constraints to describe allowed input values so that CloudFormation catches any invalid values before creating a stack.
  • For e.g. constraints for database user name with min and max length

Use AWS::CloudFormation::Init to Deploy Software Applications on Amazon EC2 Instances

  • Use AWS::CloudFormation::Init resource and the cfn-init helper script to install and configure software applications on EC2 instances

Validate Templates Before Using Them

  • Validate templates before creating or updating a stack
  • Validating a template helps catch syntax and some semantic errors, such as circular dependencies, before AWS CloudFormation creates any resources.
  • During validation, AWS CloudFormation first checks if the template is valid JSON or a valid YAML. If both checks fail, AWS CloudFormation returns a template validation error.

Managing stacks

Manage All Stack Resources Through AWS CloudFormation

  • After launching the stack, any further updates should be done through CloudFormation only.
  • Doing changes outside the stack can create a mismatch between the stack’s template and the current state of the stack resources, which can cause errors if you update or delete the stack.

Create Change Sets Before Updating Your Stacks

  • Change sets provides a preview of how the proposed changes to a stack might impact the running resources before you implement them
  • CloudFormation doesn’t make any changes to the stack until you execute the change set, allowing you to decide whether to proceed with the proposed changes or create another change set.

Use Stack Policies

  • Stack policies help protect critical stack resources from unintentional updates that could cause resources to be interrupted or even replaced
  • During a stack update, you must explicitly specify the protected resources that you want to update; otherwise, no changes are made to protected resources

Use AWS CloudTrail to Log AWS CloudFormation Calls

  • AWS CloudTrail tracks anyone making AWS CloudFormation API calls in the AWS account.
  • API calls are logged whenever anyone uses the AWS CloudFormation API, the AWS CloudFormation console, a back-end console, or AWS CloudFormation AWS CLI commands.
  • Enable logging and specify an Amazon S3 bucket to store the logs.

Use Code Reviews and Revision Controls to Manage Your Templates

  • Using code reviews and revision controls help track changes between different versions of your templates and changes to stack resources
  • Maintaining history can help revert the stack to a certain version of the template.

AWS Certification Exam Practice Questions

  • Questions are collected from Internet and the answers are marked as per my knowledge and understanding (which might differ with yours).
  • AWS services are updated everyday and both the answers and questions might be outdated soon, so research accordingly.
  • AWS exam questions are not updated to keep up the pace with AWS updates, so even if the underlying feature has changed the question might not be updated
  • Open to further feedback, discussion and correction.
  1. A company has deployed their application using CloudFormation. They want to update their stack. However, they want to understand how the changes will affect running resources before implementing the updated. How can the company achieve the same?
    1. Use CloudFormation Validate Stack feature
    2. Use CloudFormation Dry Run feature
    3. Use CloudFormation Stage feature
    4. Use CloudFormation Change Sets feature
  2. You have multiple similar three-tier applications and have decided to use CloudFormation to maintain version control and achieve automation. How can you best use CloudFormation to keep everything agile and maintain multiple environments while keeping cost down?
    1. Create multiple templates in one CloudFormation stack.
    2. Combine all resources into one template for version control and automation.
    3. Use CloudFormation custom resources to handle dependencies between stacks
    4. Create separate templates based on functionality, create nested stacks with CloudFormation.
  3. You are working as an AWS DevOps admins for your company. You are in-charge of building the infrastructure for the company’s development teams using CloudFormation. The template will include building the VPC and networking components, installing a LAMP stack and securing the created resources. As per the AWS best practices what is the best way to design this template?
    1. Create a single CloudFormation template to create all the resources since it would be easier from the maintenance perspective.
    2. Create multiple CloudFormation templates based on the number of VPC’s in the environment.
    3. Create multiple CloudFormation templates based on the number of development groups in the environment.
    4. Create multiple CloudFormation templates for each set of logical resources, one for networking, and the other for LAMP stack creation.

References

Google Cloud – Associate Cloud Engineer Certification learning path

Google Cloud Certified - Associate Cloud Engineer

Google Cloud – Associate Cloud Engineer certification exam is basically for one who works day-in day-out with the Google Cloud Services. It targets an Cloud Engineer who deploys applications, monitors operations, and manages enterprise solutions. The exam makes sure it covers gamut of services and concepts. Although, the exam is not that tough and time available of 2 hours a quite plenty, if you well prepared.

Quick summary of the exam

  • Wide range of Google Cloud services and what they actually do. It focuses heavily on IAM, Compute, Storage. There is little bit of Network but hardly any data services.
  • Hands-on is a must. Covers Cloud SDK commands and Console operations that you would use for day-to-day work. If you have not worked on GCP before make sure you do lot of labs else you would be absolute clueless for some of the questions and commands
  • Tests are updated for the latest enhancements. There are no reference of Google Container Engine and everything was Google Kubernetes Engine, covers Cloud Functions, Cloud Spanner.
  • Once again be sure that NO Online Course or Practice tests is going to cover all. I did LinuxAcademy which covered maybe 60-70%, but hands-on or practical knowledge is MUST

The list of topics is quite long, but something that you need to be sure to cover are

  • General Services
    • Billing
      • understand how billing works. Monthly vs Threshold and which has priority
      • how to change a billing account for a project and what roles you need. Hint – Project Owner and Billing Administrator for the billing account
    • Cloud SDK
      • understand gcloud commands esp. when dealing with
        • configurations i.e. gcloud config
          • activate profiles or set project and accounts
        • app engine i.e gcloud iam
          • check roles
        • deployment manager i.e. gcloud deployment-manager
  • Network Services
    • Virtual Private Cloud
      • Create a Custom Virtual Private Cloud (VPC), subnets and host applications within them Hint VPC spans across region
      • Understand how Firewall rules works and how they are configured. Hint – Focus on Network Tags.
      • Understand the concept internal and external IPs and difference between static and ephemeral IPs
    • Load Balancer
  • Identity Services
    • Cloud IAM 
      • provides administrators the ability to manage cloud resources centrally by controlling who can take what action on specific resources.
      • Understand how IAM works and how rules apply esp. the hierarchy from Organization -> Folder -> Project -> Resources
      • Understand the difference between Primitive, Pre-defined and Custom roles and their use cases
      • Need to know and understand the roles for the following services atleast
        • Cloud Storage – Admin vs Creator vs Viewer
        • Compute Engine – Admin vs Instance Admin
        • Spanner – Viewer vs Database User
        • BigQuery – User vs JobUser
      • Know how to copy roles to different projects or organization. Hint – gcloud iam roles copy
      • Know how to use service accounts with applications
  • Compute Services
    • Make sure you know all the compute services Google Compute Engine, Google App Engine and Google Kubernetes Engine, they are heavily covered in the exam.
    • Google Compute Engine
      • Google Compute Engine is the best IaaS option for compute and provides fine grained control
      • Make sure you know how to create a GCE, connect to it using Cloud shell or ssh keys
      • Make sure you know the difference between backups and images and how to create the same
      • Understand how you can recreate instance in different zones and regions
      • Know difference between managed vs unmanaged instance groups and auto-healing feature
      • Understand Preemptible VMs and their use cases.
      • know how to upgrade an instance without downtime. HINT – live migration.
      • In case of any issues or errors, how to debug the same
    • Google App Engine
      • Google App Engine is mainly the best option for PaaS with platforms supported and features provided.
      • Deploy an application with App Engine and understand how versioning and rolling deployments can be done
      • Understand how to keep auto scaling and traffic splitting and migration.
      • Know App Engine is a regional resource and understand the steps to migrate or deploy application to different region and project.
    • Google Kubernetes Engine
      • Google Container Engine is now officially Google Kubernetes Engine and the questions refer to the same
      • Google Kubernetes Engine, powered by the open source container scheduler Kubernetes, enables you to run containers on Google Cloud Platform.
      • Kubernetes Engine takes care of provisioning and maintaining the underlying virtual machine cluster, scaling your application, and operational logistics such as logging, monitoring, and cluster health management.
      • Be sure to Create a Kubernetes Cluster and configure it to host an application
      • Understand how to make the cluster auto repairable and upgradable. Hint – Node auto-upgrades and auto-repairing feature
      • Very important to understand where to use gcloud commands (to create a cluster) and kubectl commands (manage the cluster components)
      • Very important to understand how to increase cluster size and enable autoscaling for the cluster
      • know how to manage secrets like database passwords
  • Storage Services
    • Understand each storage service options and their use cases.
    • Cloud Storage
      • cost-effective object storage for an unstructured data.
      • very important to know the different classes and their use cases esp. Regional and Multi-Regional (frequent access), Nearline (monthly access) and Coldline (yearly access)
      • Understand life cycle management. HINT – Changes are in accordance to object creation date
      • Understand Signed URL to give temporary access and the users do not need to be GCP users
      • Understand permissions – IAM vs ACLs (fine grained control)
    • Relational Databases
      • Know Cloud SQL and Cloud Spanner
      • Cloud SQL
        • is a fully-managed service that provides MySQL and PostgreSQL only.
        • limited to 10TB and is a regional service.
        • know the difference between Failover and Read replicas
        • know how to perform Point-In-Time recovery. Hint – required binary logging and backups
      • Cloud Spanner
        • is a fully managed, mission-critical relational database service.
        • provides a scalable online transaction processing (OLTP) database with high availability and strong consistency at global scale.
        • globally distributed and can scale and handle more than 10TB.
        • not a direct replacement and would need migration
      • There are no direct options for Microsoft SQL Server or Oracle yet.
    • Data Warehousing
      • BigQuery
        • provides scalable, fully managed enterprise data warehouse (EDW) with SQL and fast ad-hoc queries.
        • Remember it is most suitable for historical analysis.
        • know how to perform a preview or dry run. Hint – price is determined by bytes read not bytes returned.
  • Data Services
    • Although there were only a couple of reference of big data services in the exam, it is important to know (DO NOT DEEP DIVE) the Big Data stack (esp. IoT gateway, Pub/Sub, Bigtable vs BigQuery) to understand which service fits the different layers of ingest, store, process, analytics, use
      • Cloud Storage as the medium to store data as data lake
      • Cloud Pub/Sub as the messaging service to capture real time data esp. IoT
      • Cloud Pub/Sub is designed to provide reliable, many-to-many, asynchronous messaging between applications esp. real time IoT data capture
      • Cloud Dataflow to process, transform, transfer data and the key service to integrate store and analytics.
      • Cloud BigQuery for storage and analytics. Remember BigQuery provides the same cost-effective option for storage as Cloud Storage
      • Cloud Dataprep to clean and prepare data. Hint – It can be used anomaly detection.
      • Cloud Dataproc to handle existing Hadoop/Spark jobs. Hint – Use it to replace existing hadoop infra.
      • Cloud Datalab is an interactive tool for exploration, transformation, analysis and visualization of your data on Google Cloud Platform
  • Monitoring
    • Google Stackdriver
      • provides everything from monitoring, alert, error reporting, metrics, diagnostics, debugging, trace.
      • remember audits are mainly checking Stackdriver
  • DevOps services
    • Deployment Manager 
    • Cloud Launcher (Marketplace)
      • provides a way to launch common software packages e.g. Jenkins or WordPress and stacks on Google Compute Engine with just a few clicks like a prepackaged solution.
      • It can help minimize deployment time and can be used without any knowledge about the product

Resources

Google Cloud – Professional Data Engineer Certification learning path

After completing my Google Cloud – Professional Cloud Architect certification exam, I was looking into the Google Cloud – Professional Data Engineer exam and luckily Google Cloud was doing a pilot for their latest updated Professional Data Engineer certification exam. I applied for the free pilot and had a chance to appear for the exam. The pilot exam was 4 hours – 95 questions (as compared to 2 hrs – 50 questions). The results would be out in March 2019, but I can assure the overall exam is quite exhaustive. Once again, the exam covers not only the gamut of services and concepts but also the focus on logical thinking and practical experience.

Quick summary of the exam

  • Wide range of Google Cloud data services and what they actually do. It includes Storage, and a LOTS of Data services
  • Nothing much on Compute and Network is covered
  • Questions sometimes tests your logical thinking rather than any concept regarding Google Cloud.
  • Hands-on, if you have not worked on GCP before make sure you do lots of labs else you would be absolute clueless for some of the questions and commands
  • Tests are updated for the latest enhancements.
  • Pilot exam does not cover the cases studies. But given my Professional Cloud Architect exam experience, make sure you cover the case studies before hand.
  • Be sure that NO Online Course or Practice tests is going to cover all. I did Coursera, LinuxAcademy which is really vast, but hands-on or practical knowledge is MUST.

The list of topics is quite long, but something that you need to be sure to cover are

  • Identity Services
    • Cloud IAM 
      • provides administrators the ability to manage cloud resources centrally by controlling who can take what action on specific resources.
      • Understand how IAM works and how rules apply esp. the hierarchy from Organization -> Folder -> Project -> Resources
      • Understand IAM Best practices
      • Make sure you know the BigQuery Access roles
  • Storage Services
    • Understand each storage service options and their use cases.
    • Cloud Storage
      • cost-effective object storage for an unstructured data.
      • very important to know the different classes and their use cases esp. Regional and Multi-Regional (frequent access), Nearline (monthly access) and Coldline (yearly access)
      • Understand Signed URL to give temporary access and the users do not need to be GCP users
      • Understand permissions – IAM vs ACLs (fine grained control)
    • Relational Databases
      • Know Cloud SQL and Cloud Spanner
      • Cloud SQL
        • is a fully-managed service that provides MySQL and PostgreSQL only.
        • Limited to 10TB and is a regional service.
      • Cloud Spanner
        • is a fully managed, mission-critical relational database service.
        • provides a scalable online transaction processing (OLTP) database with high availability and strong consistency at global scale.
        • globally distributed and can scale and handle more than 10TB.
        • not a direct replacement and would need migration
      • There are no direct options for Microsoft SQL Server or Oracle yet.
    • NoSQL
      • Know Cloud Datastore and BigTable
      • Datastore
        • provides document database for web and mobile applications. Datastore is not for analytics
        • Understand Datastore indexes and how to update indexes for Datastore
      • Bigtable
        • provides column database suitable for both low-latency single-point lookups and precalculated analytics
        • understand Bigtable is not for long term storage as it is quite expensive
        • know the differences with HBase
        • Know how to measure performance and scale
    • Data Warehousing
      • BigQuery
        • provides scalable, fully managed enterprise data warehouse (EDW) with SQL and fast ad-hoc queries.
        • Remember it is most suitable for historical analysis.
        • know how to access control tables, columns within tables and query results (hint – Authorized View)
        • Be sure to cover the Best Practices including key strategy, cost optimization, partitioning and clustering
  • Data Services
    • Obviously there is lots of Data and Just Data
    • Know the Big Data stack and understand which service fits the different layers of ingest, store, process, analytics, use
    • Cloud Storage
      • as the medium to store data as data lake
      • understand what class is the best suited and which one provides geo-redundancy.
    • Cloud Pub/Sub
      • as the messaging service to capture real time data esp. IoT
    • Cloud Pub/Sub
      • is designed to provide reliable, many-to-many, asynchronous messaging between applications esp. real time IoT data capture
      • how it compares to Kafka
    • Cloud Dataflow
      • to process, transform, transfer data and the key service to integrate store and analytics.
      • know how to improve a Dataflow performance
      • Google expects you to know the Apache Beam features as well
    • Cloud BigQuery
      • for storage and analytics. Remember BigQuery provides the same cost-effective option for storage as Cloud Storage
      • understand how BigQuery Streaming works
      • know BigQuery limitations esp. with updates and inserts
    • Cloud Dataprep
      • to clean and prepare data. It can be used anomaly detection.
      • does not need any programming language knowledge and can be done through graphical interface
      • be sure to know or try hands-on on a dataset
    • Cloud Dataproc
      • to handle existing Hadoop/Spark jobs
      • you need to know how to improve the performance of the Hadoop cluster as well :). Know how to configure the hadoop cluster to use all the cores (hint- spark executor cores) and handle out of memory errors (hint – executor memory)
      • how to install other components (hint – initialization actions)
    • Cloud Datalab
      • is an interactive tool for exploration, transformation, analysis and visualization of your data on Google Cloud Platform
      • based on Jupyter
    • Cloud Composer
      • fully managed workflow orchestration service based on Apache Airflow
      • pipelines are configured as directed acyclic graphs (DAGs)
      • workflow lives on-premises, in multiple clouds, or fully within GCP.
      • provides ability to author, schedule, and monitor your workflows in a unified manner
  • Machine Learning
    • Google expects the Data Engineer to surely know some of the Data scientists stuff
    • Understand the different algorithms
      • Supervised Learning (labelled data)
        • Classification (for e.g. Spam or Not)
        • Regression (for e.g. Stock or House prices)
      • Unsupervised Learning (Unlabelled data)
        • Clustering (for e.g. categories)
      • Reinforcement Learning
    • Know Cloud ML with Tensorflow
    • Know all the Cloud AI products which include
      • Cloud Vision
      • Cloud Natural Language
      • Cloud Speech-to-Text
      • Cloud Video Intelligence
    • Cloud AutoML products, which can help you get started without much machine learning experience
  • Monitoring
    • Google Stackdriver provides everything from monitoring, alert, error reporting, metrics, diagnostics, debugging, trace.
      remember audits are mainly checking Stackdriver
  • Security Services
    • Data Loss Prevention API to handle sensitive data esp. redaction of PII data.
    • understand Encryption techniques
  • Other Services
    • Storage Transfer Service allows import of large amounts of online data into Google Cloud Storage, quickly and cost-effectively. Online data is the key here as it supports AWS S3, HTTP/HTTPS and other GCS buckets. If the data is on-premises you need to use gsutil command
    • Transfer Appliance to transfer large amounts of data quickly and cost-effectively into Google Cloud Platform. Check for the data size and it would be always compared with Google Transfer Service or gsutil commands.
    • BigQuery Data Transfer Service to integrate with third-party services and load data into BigQuery

Resources

Google Cloud – Professional Cloud Architect Certification learning path

Google Cloud – Professional Cloud Architect certification exam is one of the toughest exam I have appeared for. It can surely be compared inline with the AWS Solution Architect/DevOps Professional exams. However, the gamut of services and concepts it tests your knowledge on is really vast.

Quick summary of the exam

  • Wide range of Google Cloud services and what they actually do. It includes Compute, Storage, Network and even Data services
  • Questions sometimes tests your logical thinking rather than any concept regarding Google Cloud.
  • Hands-on, if you have not worked on GCP before make sure you do lots of labs else you would be absolute clueless for some of the questions and commands
  • Tests are updated for the latest enhancements. There are no reference of Google Container Engine and everything was Google Kubernetes Engine, covers Cloud Functions, Cloud Spanner.
  • Make sure you cover the case studies before hand. I got around 15 questions (almost 5 per case study) and it can really be a savior for you in the exams.
  • Be sure that NO Online Course or Practice tests is going to cover all. I did LinuxAcademy which is really vast, but hands-on or practical knowledge is MUST.

The list of topics is quite long, but something that you need to be sure to cover are

  • Identity Services
    • Cloud IAM 
      • provides administrators the ability to manage cloud resources centrally by controlling who can take what action on specific resources.
      • Understand how IAM works and how rules apply esp. the hierarchy from Organization -> Folder -> Project -> Resources
      • How do you use on-premises authentication provider? Google Cloud Directory Sync (GCDS)
  • Compute Services
    • Make sure you know all the compute services Google Compute Engine, Google App Engine and Google Kubernetes Engine. You need to be sure to know the pros and cons and the use cases that you should use them.
    • Google Compute Engine
      • Google Compute Engine is the best IaaS option for compute and provides fine grained control
      • Make sure you know how to create a GCE, connect to it using Cloud shell or ssh keys
      • Make sure you know the difference between backups and images and how to create the same
      • Understand how you can recreate instance in different zones and regions
      • Understand the pricing and discounts model Hint – Sustained (automatic upto 30%) vs Committed (1 to 3 yrs) discounts.
      • Understand Preemptible VMs and their use cases.
      • Managed instance groups are covered heavily the exam, as they provide the key auto-scaling capability. Hint you need to create an Instance template and associate it with Instance group
      • Understand how migration or traffic splitting with Managed instance groups works Hint – rolling updates & deployments
      • In case of any issues or errors, how to debug the same
    • Google App Engine
      • Google App Engine is mainly the best option for PaaS with platforms supported and features provided.
      • Understand the key differences between Standard and Flexible App Engine. Hint – network did not work for Standard, so VPN connections did not work and you would need to use Flexible environment.
      • Deploy an application with App Engine and understand how versioning and rolling deployments can be done
    • Google Kubernetes Engine
      • Google Container Engine is now officially Google Kubernetes Engine and the questions refer to the same
      • Google Kubernetes Engine, powered by the open source container scheduler Kubernetes, enables you to run containers on Google Cloud Platform.
      • Kubernetes Engine takes care of provisioning and maintaining the underlying virtual machine cluster, scaling your application, and operational logistics such as logging, monitoring, and cluster health management.
      • Be sure to Create a Kubernetes Cluster and configure it to host an application
      • Very important to understand where to use gcloud commands (to create a cluster) and kubectl commands (manage the cluster components)
    • Cloud Functions
      • is a lightweight, event-based, asynchronous compute solution that allows you to create small, single-purpose functions that respond to cloud events without the need to manage a server or a runtime environment.
      • Remember that Cloud Functions is serverless and scales from zero to scale and back to zero as the demand changes.
  • Network Services
    • Virtual Private Cloud
      • Create a Custom Virtual Private Cloud (VPC), subnets and host applications within them Hint VPC spans across region
      • Understand how Firewall rules works and how they are configured. Hint – Focus on Network Tags.
      • Understand the concept of shared VPC which allows for access to resources using internal IPs
      • Understand VPC Peering and Private Google Access use cases
    • On-premises connectivity
      • Cloud VPN and Interconnect are 2 components which help you connect to on-premises data center.
      • Understand limitations of Cloud VPN esp. 1.5Gbps limit. How it can be improved with multiple tunnels.
      • Understand what are the requirements to setup Cloud VPN. Hint – Cloud Router is required for BGP.
      • Know Interconnect as the reliable high speed, low latency and dedicated bandwidth options.
    • Cloud Load Balancer (GCLB)
      • Google Cloud Load Balancing provides scaling, high availability, and traffic management for your internet-facing and private applications.
      • Understand Google Load Balancing options and their use cases esp. which is global and internal and what protocols they support.
  • Storage Services
    • Understand each storage service options and their use cases.
    • Persistent disks
      • attached to the Compute Engines, provide fast access however are limited in scalability, availability and scope.
      • Remember performance depends on the size of the disk
    • Cloud Storage
      • cost-effective object storage for an unstructured data.
      • very important to know the different classes and their use cases esp. Regional and Multi-Regional (frequent access), Nearline (monthly access) and Coldline (yearly access)
      • Understand how encryption works
      • Understand Signed URL to give temporary access and the users do not need to be GCP users
      • Understand permissions – IAM vs ACLs (fine grained control)
    • Relational Databases
      • Know Cloud SQL and Cloud Spanner
      • Cloud SQL
        • is a fully-managed service that provides MySQL and PostgreSQL only.
        • Limited to 10TB and is a regional service.
      • Cloud Spanner
        • is a fully managed, mission-critical relational database service.
        • provides a scalable online transaction processing (OLTP) database with high availability and strong consistency at global scale.
        • globally distributed and can scale and handle more than 10TB.
        • not a direct replacement and would need migration
      • There are no direct options for Microsoft SQL Server or Oracle yet.
    • NoSQL
      • Know Cloud Datastore and BigTable
      • Datastore
        • provides document database for web and mobile applications. Datastore is not for analytics
        • Understand Datastore indexes and how to update indexes for Datastore
        • Can be configured Multi-regional and regional
      • Bigtable
        • provides column database suitable for both low-latency single-point lookups and precalculated analytics
        • understand Bigtable is not for long term storage as it is quite expensive
    • Data Warehousing
      • BigQuery
        • provides scalable, fully managed enterprise data warehouse (EDW) with SQL and fast ad-hoc queries.
        • Remember it is most suitable for historical analysis.
    • MemoryStore and Firebase did not feature in any of the questions
  • Data Services
    • Although there is a different certification for Data Engineer, the Cloud Architect does cover data services. Data services are also part of the use cases so be sure to know about them
    • Know the Big Data stack and understand which service fits the different layers of ingest, store, process, analytics, use
    • Key Services which need to be mainly covered are –
      • Cloud Storage as the medium to store data as data lake
      • Cloud Pub/Sub as the messaging service to capture real time data esp. IoT
      • Cloud Pub/Sub is designed to provide reliable, many-to-many, asynchronous messaging between applications esp. real time IoT data capture
      • Cloud Dataflow to process, transform, transfer data and the key service to integrate store and analytics.
      • Cloud BigQuery for storage and analytics. Remember BigQuery provides the same cost-effective option for storage as Cloud Storage
      • Cloud Dataprep to clean and prepare data. Hint – It can be used anomaly detection.
      • Cloud Dataproc to handle existing Hadoop/Spark jobs. Hint – Use it to replace existing hadoop infra.
      • Cloud Datalab is an interactive tool for exploration, transformation, analysis and visualization of your data on Google Cloud Platform
  • Monitoring
    • Google Stackdriver
      • provides everything from monitoring, alert, error reporting, metrics, diagnostics, debugging, trace.
      • remember audits are mainly checking Stackdriver
  • DevOps services
    • Deployment Manager is Infrastructure as Code
    • Cloud Source Repositories provides source code repository with Git version control to support collaborative development
    • Container Registry is a private Docker image storage system on Google Cloud Platform. Images are immutable.
    • Cloud Build is a service that executes your builds on Google Cloud Platform infrastructure.
    • Cloud Launcher provides a way to launch common software packages e.g. Jenkins or WordPress and stacks on Google Compute Engine with just a few clicks like a prepackaged solution. It can help minimize deployment time
  • Security Services
    • Cloud Security Scanner is a web application security scanner that enables developers to easily check for a subset of common web application vulnerabilities in websites built on App Engine and Compute Engine.
    • Data Loss Prevention API to handle sensitive data esp. redaction of PII data.
    • Focus on PCI-DSS, how to handle to same. Remember, GCP services are PCI-DSS complaint, however you need to make sure for your applications and hosting to be inline with PCI-DSS.
    • Same concept as PCI-DSS applies to GDPR as well
  • Other Services
    • Storage Transfer Service allows import of large amounts of online data into Google Cloud Storage, quickly and cost-effectively. Online data is the key here as it supports AWS S3, HTTP/HTTPS and other GCS buckets. If the data is on-premises you need to use gsutil command
    • Transfer Appliance to transfer large amounts of data quickly and cost-effectively into Google Cloud Platform. Check for the data size and it would be always compared with Google Transfer Service or gsutil commands.
    • Spinnaker is an open source, multi-cloud, continuous delivery platform and does appear in answer options. So be sure to know about it.
    • Jenkins for Continuous Integration and Continuous Delivery.
  • Case Studies

Resources

Google Cloud Certified – Professional Cloud Architect exam assesses your ability to

Section 1: Designing and planning a cloud solution architecture

  • 1.1 Designing a solution infrastructure that meets business requirements. Considerations include:
    • business use cases and product strategy
    • cost optimization
    • supporting the application design
    • integration
    • movement of data
    • tradeoffs
    • build, buy or modify
    • success measurements (e.g., Key Performance Indicators (KPI), Return on Investment (ROI), metrics)
    • Compliance and observability
  • 1.2 Designing a solution infrastructure that meets technical requirements. Considerations include:
    • high availability and failover design
    • elasticity of cloud resources
    • scalability to meet growth requirements
  • 1.3 Designing network, storage, and compute resources. Considerations include:
    • integration with on premises/multi-cloud environments
      Cloud native networking (VPC, peering, firewalls, container networking)
    • identification of data processing pipeline
    • matching data characteristics to storage systems
    • data flow diagrams
    • storage system structure (e.g., Object, File, RDBMS, NoSQL, NewSQL)
    • mapping compute needs to platform products
  • 1.4 Creating a migration plan (i.e., documents and architectural diagrams). Considerations include:
    • integrating solution with existing systems
    • migrating systems and data to support the solution
    • licensing mapping
    • network and management planning
    • testing and proof-of-concept
  • 1.5 Envisioning future solution improvements. Considerations include:
    • cloud and technology improvements
    • business needs evolution
    • evangelism and advocacy

Section 2: Managing and provisioning solution Infrastructure

  • 2.1 Configuring network topologies. Considerations include:
    • extending to on-premise (hybrid networking)using VPN or Interconnect
    • extending to a multi-cloud environment which may include GCP to GCP communication
    • security
    • data protection
  • 2.2 Configuring individual storage systems. Considerations include:
    • data storage allocation
    • data processing/compute provisioning
    • security and access management
    • network configuration for data transfer and latency
    • data retention and data lifecycle management
    • data growth management
  • 2.3 Configuring compute systems. Considerations include:
    • compute system provisioning
    • compute volatility configuration (preemptible vs. standard)
    • network configuration for compute nodes
    • infrastructure provisioning technology configuration (e.g. Chef/Puppet/Ansible/Terraform)
    • container orchestration (e.g. Kubernetes)

Section 3: Designing for security and compliance

  • 3.1 Designing for security. Considerations include:
    • Identity and Access Management (IAM)
    • Resource hierarchy (organizations, folders, projects)
    • data security (key management, encryption)
    • penetration testing
    • Separation of Duties (SoD)
    • security controls
    • Managing customer-supplied encryption keys with Cloud KMS
  • 3.2 Designing for legal compliance. Considerations include:
    • legislation (e.g., Health Insurance Portability and Accountability Act (HIPAA), Children’s Online Privacy Protection Act (COPPA), etc.)
      audits (including logs)
    • certification (e.g., Information Technology Infrastructure Library (ITIL) framework)

Section 4: Analyzing and optimizing technical and business processes

  • 4.1 Analyzing and defining technical processes. Considerations include:
    • Software Development Lifecycle Plan (SDLC)
    • continuous integration / continuous deployment
    • troubleshooting / post mortem analysis culture
    • testing and validation
    • IT enterprise process (e.g. ITIL)
    • business continuity and disaster recovery
  • 4.2 Analyzing and defining business processes. Considerations include:
    • stakeholder management (e.g. Influencing and facilitation)
    • change management
    • team assessment / skills readiness
    • decision making process
    • customer success management
    • cost optimization / resource optimization (Capex / Opex)
  • 4.3 Developing procedures to test resilience of solution in production (e.g., DiRT and Simian Army)

Section 5: Managing implementation

  • 5.1 Advising development/operation team(s) to ensure successful deployment of the solution. Considerations include:
    • application development
    • API best practices
    • testing frameworks (load/unit/integration)
    • data and system migration tooling
  • 5.2 Interacting with Google Cloud using GCP SDK (gcloud, gsutil and bq). Considerations include:
    • local installation
    • Google Cloud Shell

Section 6: Ensuring solution and operations reliability

  • 6.1 Monitoring/Logging/Alerting solution
  • 6.2 Deployment and release management
  • 6.3 Supporting operational troubleshooting
  • 6.4 Evaluating quality control measures

Case Studies

  • Mountkirk Games
  • Dress4Win
  • TerramEarth

AWS Certified SysOps Administrator – Associate (SOA-C01) Exam Learning Path

AWS Certified SysOps Administrator – Associate (SOA-C01) Exam Learning Path

AWS Certified SysOps Administrator – Associate (SOA-C01) exam is the latest AWS exam and has already replaced the old SysOps Administrator – Associate exam from 24th Sept 2018. It basically validates

  • Deploy, manage, and operate scalable, highly available, and fault tolerant systems on AWS
  • Implement and control the flow of data to and from AWS
  • Select the appropriate AWS service based on compute, data, or security requirements
  • Identify appropriate use of AWS operational best practices
  • Estimate AWS usage costs and identify operational cost control mechanisms
  • Migrate on-premises workloads to AWS

Refer AWS Certified SysOps – Associate Exam Guide Sep 18

AWS Certified SysOps Administrator - Associate Content Outline

AWS Certified SysOps Administrator – Associate (SOA-C01) Exam Summary

  • AWS Certified SysOps Administrator – Associate exam is quite different from the previous one with more focus on the error handling, deployment, monitoring.
  • AWS Certified SysOps Administrator – Associate exam covers a lot of latest AWS services like ALB, Lambda, AWS Config, AWS Inspector, AWS Shield while focusing majorly on other services like CloudWatch, Metrics from various services, CloudTrail.
  • Be sure to cover the following topics
    •  Monitoring & Management Tools
      • Understand CloudWatch monitoring to provide operational transparency
        • Know which EC2 metrics it can track (disk, network, CPU, status checks) and which would need custom metrics (memory, disk swap, disk storage etc.)
        • Know ELB monitoring
          • Classic Load Balancer metrics SurgeQueueLength and SpilloverCount
          • Reasons for 4XX and 5XX errors
      • Understand CloudTrail for audit and governance
      • Understand AWS Config and its use cases
      • Understand AWS Systems Manager and its various services like parameter store, patch manager
      • Understand AWS Trusted Advisor and what it provides
      • Very important to understand AWS CloudWatch vs AWS CloudTrail vs AWS Config
      • Very important to understand Trust Advisor vs Systems manager vs Inspector
      • Know Personal Health Dashboard & Service Health Dashboard
      • Deployment tools
        • Know AWS OpsWorks and its ability to support chef & puppet
        • Know Elastic Beanstalk and its advantages
        • Understand AWS CloudFormation
          • Know stacks, templates, nested stacks
          • Know how to wait for resources setup to be completed before proceeding esp. cfn-signal
          • Know how to retain resources (RDS, S3), prevent rollback in case of a failure
    • Networking & Content Delivery
      • Understand VPC in depth
        • Understand the difference between
          • Bastion host – allow access to instances in private subnet
          • NAT – route traffic from private subnets to internet
          • NAT instance vs NAT Gateway
          • Internet Gateway – Access to internet
          • Virtual Private Gateway – Connectivity between on-premises and VPC
          • Egress-Only Internet Gateway – relevant to IPv6 only to allow egress traffic from private subnet to internet, without allowing ingress traffic
        • Understand
        • Understand how VPC Peering works and limitations
        • Understand VPC Endpoints and supported services
        • Ability to debug networking issues like EC2 not accessible, EC2 instances not reachable, Instances in subnets not able to communicate with others or Internet.
      • Understand Route 53 and Routing Policies and their use cases
        • Focus on Weighted, Latency routing policies
      • Understand VPN and Direct Connect and their use cases
      • Understand CloudFront and use cases
      • Understand ELB, ALB and NLB and what features they provide like
        • ALB provides content and path routing
        • NLB provides ability to give static IPs to load balancer.
    • Compute
      • Understand EC2 in depth
        • Understand EC2 instance types
        • Understand EC2 purchase options esp. spot instances and improved reserved instances options.
        • Understand how IO Credits work and T2 burstable performance and T2 unlimited
        • Understand EC2 Metadata & Userdata. Whats the use of each? How to look up instance data after it is launched.
        • Understand EC2 Security. 
          • How IAM Role work with EC2 instances
          • IAM Role can now be attached to stopped and runnings instances
        • Understand AMIs and remember they are regional and how can they be shared with others.
        • Troubleshoot issues with launching EC2 esp. RequestLimitExceeded, InstanceLimitExceeded etc.
        • Troubleshoot connectivity, lost ssh keys issues
      • Understand Auto Scaling
      • Understand Lambda and its use cases
      • Understand Lambda with API Gateway
    • Storage
    • Databases
    • Security
      • Understand IAM as a whole
      • Understand KMS for key management and envelope encryption
      • Understand CloudHSM and KMS vs CloudHSM esp. support for symmetric and asymmetric keys
      • Know AWS Inspector and its use cases
      • Know AWS GuardDuty as managed threat detection service. Will help eliminate as the option
      • Know AWS Shield esp. the Shield Advanced option and the features it provides
      • Know WAF as Web Traffic Firewall
      • Know AWS Artifact as on-demand access to compliance reports
    • Integration Tools
      • Understand SQS as message queuing service and SNS as pub/sub notification service
        • Focus on SQS as a decoupling service
        • Understand SQS FIFO, make sure you know the differences between standard and FIFO
      • Understand CloudWatch integration with SNS for notification
    • Cost management

AWS Certified SysOps Administrator – Associate (SOA-C01) Exam Resources

AWS Cloud Computing Whitepapers

AWS Certified SysOps Administrator – Associate (SOA-C01) Exam Contents

Domain 1: Monitoring and Reporting

    1. Create and maintain metrics and alarms utilizing AWS monitoring services
    1. Recognize and differentiate performance and availability metrics
  1. Perform the steps necessary to remediate based on performance and availability metrics

Domain 2: High Availability

    1. Implement scalability and elasticity based on use case
  1. Recognize and differentiate highly available and resilient environments on AWS

Domain 3: Deployment and Provisioning

    1. Identify and execute steps required to provision cloud resources
  1. Identify and remediate deployment issues

Domain 4: Storage and Data Management

    1. Create and manage data retention
  1. Identify and implement data protection, encryption, and capacity planning needs

Domain 5: Security and Compliance

    1. Implement and manage security policies on AWS
    1. Implement access controls when using AWS
  1. Differentiate between the roles and responsibility within the shared responsibility model

Domain 6: Networking

    1. Apply AWS networking features
    1. Implement connectivity services of AWS
  1. Gather and interpret relevant information for network troubleshooting

Domain 7: Automation and Optimization

  1. Use AWS services and features to manage and assess resource utilization
  2. Employ cost-optimization strategies for efficient resource utilization
  3. Automate manual or repeatable process to minimize management overhead

AWS Certified Developer – Associate DVA-C01 Exam Learning Path

AWS Certified Developer – Associate DVA-C01 Exam Learning Path

AWS Certified Developer – Associate DVA-C01 exam is the latest AWS exam and would replace the old Developer – Associate exam. It basically validates

  • Demonstrate an understanding of core AWS services, uses, and basic AWS architecture best practices.
  • Demonstrate proficiency in developing, deploying, and debugging cloud-based applications using AWS.

Refer AWS Certified Developer – Associate (Released June 2018) Exam Blue Print

AWS Certified Developer - Associate June 2018 Domains

AWS Certified Developer – Associate DVA-C01 Summary

  • AWS Certified Developer – Associate DVA-C01 exam is quite different from the previous one with more focus on the hands-on development and deployment concepts rather then just the architectural concepts
  • AWS Certified Developer – Associate DVA-C01 exam covers a lot of latest AWS services like Lambda, X-Ray while focusing majorly on other services like DynamoDB, Elastic Beanstalk, S3, EC2
  • Be sure to cover the following topics
    • Compute
      • Understand what AWS services you can use to build a serverless architecture?
      • Make sure you know and understand Lambda and serverless architecture, its features and use cases.
      • Know Lambda limits for e.g. execution time, deployable zipped and unzipped package limit
      • Be sure to know how to deploy, package using Lambda.
      • Understand tracing of Lambda functions using X-Ray
      • Understand integration of Lambda with CloudWatch.
      • Understand how to handle multiple releases using Alias
      • Know AWS Step Functions to manage Lambda functions flow
      • Understand Lambda with API Gateway
      • Understand API Gateway stages, ability to cater to different environments for e.g. dev, test, prod
      • Understand EC2 as a whole
      • Understand EC2 Metadata & Userdata. Whats the use of each? How to look up instance data after it is launched.
      • Understand EC2 Security. How IAM Role work with EC2 instances.
      • Understand how does EC2 evaluates the order of credentials, when multiple are provided. Remember the order – Environment variables -> Java system properties -> Default credential profiles file -> ECS container credentials -> Instance Profile credentials
      • Know Elastic Beanstalk at a high level, what it provides and its ability to get an application running quickly
      • Understand Elastic Beanstalk configurations and deployment types with their advantages and disadvantages
    • Databases
      • Understand relational and NoSQLs data storage options which include RDS, DynamoDB and their use cases
      • Understand DynamoDB Secondary Indexes
      • Make sure you understand DynamoDB provisioned throughput for Read/Writes and its calculations
      • Make sure you understand DynamoDB Consistency Model – difference between Strongly Consistent and Eventual Consistency
      • Understand DynamoDB with its low latency performance, DAX
      • Know how to configure fine grained security for DynamoDB table, items, attributes
      • Understand DynamoDB Best Practices regarding
        • table design
        • provisioned throughput
        • Query vs Scan operations
        • improving Scan operation performance
      • Understand RDS features – Read Replicas for scalability, Multi-AZ for High Availability
      • Know ElastiCache use cases, mainly for caching performance
      • Understand ElastiCache Redis vs Memcached
    • Storage
      • Understand S3 storage option
      • Understand S3 Best Practices to improve performance for GET/PUT requests
      • Understand S3 features like different storage classes with lifecycle policies, static website hosting, versioning, Pre-Signed URLs for both upload and download, CORS
    • Security
      • Understand IAM as a whole
      • Focus on IAM role and its use case especially with EC2 instance
      • Know how to test and validate IAM policies
      • Understand IAM identity providers and federation and use cases
      • Understand how AWS Cognito works and what features it provides
      • Understand MFA and How would implement two factor authentication for your application
      • Understand KMS for key management and envelope encryption
      • Know what services support KMS
        • Remember SQS, Kinesis now provides SSE support
      • Focus on S3 with SSE, SSE-C, SSE-KMS. How they work and differ?
      • Know how can you enforce only buckets to only accept encrypted objects
      • Know various KMS encryption options encrypt, reencrypt, generateEncryptedDataKey etc
      • Know how KMS impacts performance of the services
    • Management Tools
      • Understand CloudWatch monitoring to provide operational transparency
      • Know which EC2 metrics it can track.
      • Understand CloudWatch is extendable with custom metrics
      • Understand CloudTrail for Audit
    • Integration Tools
      • Understand SQS as message queuing service and SNS as pub/sub notification service
      • Understand SQS features like visibility, long poll vs short poll
      • Focus on SQS as a decoupling service
      • AWS has released SQS FIFO, make sure you know the differences between standard and FIFO
      • Know the different development and deployment tools like CodeCommit, CodeBuild, CodeDeploy, CodePipeline
    • Networking
      • Does not cover much on networking or designing of networks, but be sure you understand VPC, Subnets, Routes, Security Groups etc.

AWS Certified Developer – Associate DVA-C01 Exam Resources

AWS Cloud Computing Whitepapers

AWS Certified Developer – Associate DVA-C01 Exam Contents

Domain 1: Deployment

    1. Deploy written code in AWS using existing CI/CD pipelines, processes, and patterns.
    1. Deploy applications using Elastic Beanstalk.
    1. Prepare the application deployment package to be deployed to AWS.
  1. Deploy serverless applications.

Domain 2: Security

    1. Make authenticated calls to AWS services.
    1. Implement encryption using AWS services.
  1. Implement application authentication and authorization.

Domain 3: Development with AWS Services

    1. Write code for serverless applications.
    1. Translate functional requirements into application design.
    1. Implement application design into application code.
  1. Write code that interacts with AWS services by using APIs, SDKs, and AWS CLI.

Domain 4: Refactoring

    1. Optimize application to best use AWS services and features.
  1. Migrate existing application code to run on AWS.

Domain 5: Monitoring and Troubleshooting

    1. Write code that can be monitored.
  1. Perform root cause analysis on faults found in testing or production.

AWS RDS Aurora – Certification

AWS Aurora

  • AWS Aurora is a relational database engine that combines the speed and reliability of high-end commercial databases with the simplicity and cost-effectiveness of open source databases.
  • Amazon Aurora (Aurora) is a fully managed, MySQL- and PostgreSQL-compatible, relational database engine i.e. applications developed with MySQL can switch to Aurora with little or no changes
  • Aurora delivers up to 5x performance of MySQL without requiring any changes to most MySQL applications
  • Aurora PostgreSQL delivers up to 3x performance of PostgreSQL.
  • RDS manages the Aurora databases, handling time-consuming tasks such as provisioning, patching, backup, recovery, failure detection and repair.
  • Based on the database usage, Aurora storage will automatically grow, from 10GB to 64TB in 10GB increments with no impact to database performance

NOTE – AWS Aurora is covered in the latest Solution Architect – Associate Feb 2018 extensively, so be sure to cover the same.

AWS Aurora Architecture

High Availability and Replication

  • Aurora provides data durability and reliability by replicating the database volume six ways across three Availability Zones in a single region
    • Aurora automatically divides the database volume into 10GB segments spread across many disks.
    • Each 10GB chunk of your database volume is replicated six ways, across three Availability Zones
  • RDS databases for e.g. MySQL, Oracle etc. have the data in a single AZ
  • Aurora is designed to transparently handle the loss of up to two copies of data without affecting database write availability and up to three copies without affecting read availability.
  • Aurora storage is also self-healing. Data blocks and disks are continuously scanned for errors and repaired automatically.
  • Aurora Replicas share the same underlying volume as the primary instance. Updates made by the primary are visible to all Aurora Replicas
  • As Aurora Replicas share the same data volume as the primary instance, there is virtually no replication lag
  • Any Aurora Replica can be promoted to become primary without any data loss and therefore can be used for enhancing fault tolerance in the event of a primary DB Instance failure.
  • To increase database availability, 1 to 15 replicas can be created in any of 3 AZs, and RDS will automatically include them in failover primary selection in the event of a database outage.

Security

  • Aurora uses SSL (AES-256) to secure the connection between the database instance and the application
  • Aurora allows database encryption using keys managed through AWS Key Management Service (KMS).
  • Encryption and decryption are handled seamlessly.
  • With Aurora encryption, data stored at rest in the underlying storage is encrypted, as are its automated backups, snapshots, and replicas in the same cluster.
  • Encryption of existing unencrypted Aurora instance is not supported. Create a new encrypted Aurora instance and migrate the data

Backup and Restore

  • Automated backups are always enabled on Aurora DB Instances.
  • Backups do not impact database performance.
  • Aurora also allows creation of manual snapshots
  • Aurora automatically maintains 6 copies of your data across 3 AZs and will automatically attempt to recover your database in a healthy AZ with no data loss.
  • If in any case the data is unavailable within Aurora storage,
    • DB Snapshot can be restored or
    • point-in-time restore operation can be performed to a new instance. Latest restorable time for a point-in-time restore operation can be up to 5 minutes in the past.
  • Restoring a snapshot creates a new Aurora DB instance
  • Deleting Aurora database deletes all the automated backups (with an option to create a final snapshot), but would not remove the manual snapshots.
  • Snapshots (including encrypted ones) can be shared with another AWS accounts

AWS Certification Exam Practice Questions

  • Questions are collected from Internet and the answers are marked as per my knowledge and understanding (which might differ with yours).
  • AWS services are updated everyday and both the answers and questions might be outdated soon, so research accordingly.
  • AWS exam questions are not updated to keep up the pace with AWS updates, so even if the underlying feature has changed the question might not be updated
  • Open to further feedback, discussion and correction.
  1. Company wants to use MySQL compatible relational database with greater performance. Which AWS service can be used?
    1. Aurora
    2. RDS
    3. SimpleDB
    4. DynamoDB
  2. An application requires a highly available relational database with an initial storage capacity of 8 TB. The database will grow by 8 GB every day. To support expected traffic, at least eight read replicas will be required to handle database reads. Which option will meet these requirements?
    1. DynamoDB
    2. Amazon S3
    3. Amazon Aurora
    4. Amazon Redshift
  3. A company is migrating their on-premise 10TB MySQL database to AWS. As a compliance requirement, the company wants to have the data replicated across three availability zones. Which Amazon RDS engine meets the above business requirement?
    1. Use Multi-AZ RDS
    2. Use RDS
    3. Use Aurora
    4. Use DynamoDB

AWS X-Ray – Certification

AWS X-Ray

  • AWS X-Ray helps developers analyze and debug production, distributed applications for e.g. built using a microservices lambda architecture
  • X-Ray provides an end-to-end view of requests as they travel through the application, and shows a map of the application’s underlying components.
  • X-Ray helps to understand how the application and its underlying services are performing to identify and troubleshoot the root cause of performance issues and errors.
  • X-Ray to analyze both applications in development and in production, from simple three-tier applications to complex microservices applications consisting of thousands of services.
  • X-Ray can be used with distributed applications of any size to trace and debug both synchronous requests and asynchronous events.
  • X-Ray can be used to track requests flowing through applications or services across multiple regions. X-Ray data is stored locally to the processed region and customers can build a solution over it to combined the data
  • Trace data sent to X-Ray is generally available for retrieval and filtering within 30 seconds of it being received by the service.
  • X-Ray stores trace data for the last 30 days. This enables you to query trace data going back 30 days.
  • Integration
    • X-Ray integrates with applications running on EC2, ECS, Lambda, and Elastic Beanstalk.
    • X-Ray SDK automatically captures metadata for API calls made to AWS services using the AWS SDK
    • X-Ray SDK provides add-ons for MySQL and PostgreSQL drivers.
    • For Elastic Beanstalk, include the language-specific X-Ray libraries in your application code.
    • Applications running on other AWS services, such as EC2 or ECS, install the X-Ray agent and instrument the application code

X-Ray Core Concepts

  • Trace
    • An X-Ray trace is a set of data points that share the same trace ID.
    • Trace helps track the request, which is assigned a unique trace id, while it navigates through services
    • Piece of information relayed by each service in the application to X-Ray is a segment, and a trace is a collection of segments.
  • Segment
    • An X-Ray segment encapsulates all the data points for a single component of the distributed application for e.g. authorization component
    • Segments include system-defined and user-defined data in the form of annotations and are composed of one or more sub-segments that represent remote calls made from the service.  for e.g. database call and its result within the overall request/response
  • Annotation
    • An X-Ray annotation is system-defined or user-defined data
    • Annotation is associated with a segment and a segment can contain multiple annotations.
    • System-defined annotations include data added to the segment by AWS services
    • User-defined annotations are metadata added to a segment by a developer
  • Errors
    • X-Ray errors are system annotations associated with a segment for a call that results in an error response.
    • Error includes the error message, stack trace, and any additional information for e.g, version to associate the error with a source file.
  • Sampling
    • X-Ray collects data for significant number of requests, instead of each request sent to an application, for performant and cost-effectiveness
    • X-Ray should not be used as an audit or compliance tool because it does not guarantee data completeness.
  • X-Ray agent
    • X-Ray agent helps collect data from log files and sends them to the X-Ray service for aggregation, analysis, and storage.
    • Agent makes it easier for you to send data to the X-Ray service, instead of using the APIs directly, and is
    • Agent is available for Amazon Linux AMI, Red Hat Enterprise Linux (RHEL), and Windows Server 2012 R2 or later operating systems.

AWS Certification Exam Practice Questions

  • Questions are collected from Internet and the answers are marked as per my knowledge and understanding (which might differ with yours).
  • AWS services are updated everyday and both the answers and questions might be outdated soon, so research accordingly.
  • AWS exam questions are not updated to keep up the pace with AWS updates, so even if the underlying feature has changed the question might not be updated
  • Open to further feedback, discussion and correction.
  1. A company is facing performance issues with their microservices architecture deployed on AWS. Which service can help them debug and analyze the issue? [CCP]
    1. AWS Inspector
    2. CodeDeploy
    3. X-Ray
    4. AWS Config

AWS DynamoDB Advanced – Certification

AWS DynamoDB Advanced Features

This post covers DynamoDB advanced features

DynamoDB Secondary Indexes

DynamoDB supports Local and Global Secondary Indexes.

Refer to My Blog Post about AWS DynamoDB Secondary Indexes

DynamoDB Cross-region Replication

  • DynamoDB cross-region replication allows identical copies (called replicas) of a DynamoDB table (called master table) to be maintained in one or more AWS regions
  • Writes to the table will be automatically propagated to all replicas
  • Cross-region replication currently supports single master mode. A single master has one master table and one or more replica tables
  • Read replicas are updated asynchronously as DynamoDB acknowledges a write operation as successful once it has been accepted by the master table. The write will then be propagated to each replica with a slight delay.
  • Cross-region replication can be helpful in scenarios
    • Efficient disaster recovery, in case a data center failure occurs.
    • Faster reads, for customers in multiple regions by delivering data faster by reading a DynamoDB table from the closest AWS data center.
    • Easier traffic management, to distribute the read workload across tables and thereby consume less read capacity in the master table.
    • Easy regional migration, by promoting a read replica to master
    • Live data migration, to replicate data and when the tables are in sync, switch the application to write to the destination region
  • Cross-region replication costing depends on
    • Provisioned throughput (Writes and Reads)
    • Storage for the replica tables.
    • Data Transfer across regions
    • Reading data from DynamoDB Streams to keep the tables in sync.
    • Cost of EC2 instances provisioned, depending upon the instance types and region, to host the replication process.
  • NOTE : Cross Region replication on DynamoDB was performed defining AWS Data Pipeline job which used EMR internally to transfer data before the DynamoDB streams and out of box cross region replication support

DynamoDB Global Tables

  • DynamoDB Global Tables is a new multi-master, cross-region replication capability of DynamoDB to support data access locality and regional fault tolerance for database workloads.
  • Applications can now perform reads and writes to DynamoDB in AWS regions around the world, with changes in any region propagated to every region where a table is replicated.
  • Global Tables help in building applications to advantage of data locality to reduce overall latency.
  • Global Tables ensures eventual consistency
  • Global Tables replicates data among regions within a single AWS account, and currently does not support cross account access

DynamoDB Streams

  • DynamoDB Streams provides a time-ordered sequence of item-level changes made to data in a table in the last 24 hours, after which they are erased
  • DynamoDB Streams maintains ordered sequence of the events per item however across item are not maintained
  • Example
    • For e.g., suppose that you have a DynamoDB table tracking high scores for a game and that each item in the table represents an individual player. If you make the following three updates in this order:
      • Update 1: Change Player 1’s high score to 100 points
      • Update 2: Change Player 2’s high score to 50 points
      • Update 3: Change Player 1’s high score to 125 points
    • DynamoDB Streams will maintain the order for Player 1 score events. However, it would not maintain the order across the players. So Player 2 score event is not guaranteed between the 2 Player 1 events
  • DynamoDB streams can be used for multi-region replication to keep other data stores up-to-date with the latest changes to DynamoDB or to take actions based on the changes made to the table
  • DynamoDB Streams APIs helps developers consume updates and receive the item-level data before and after items are changed
  • DynamoDB Streams allows read at up to twice the rate of the provisioned write capacity of the DynamoDB table
  • DynamoDB Streams have to be enabled on a per-table basis
  • DynamoDB Streams is designed so that every update made to the table will be represented exactly once in the stream
  • DynamoDB Streams is designed so that every update made to the table will be represented exactly once in the stream. No Duplicates

DynamoDB Triggers

  • DynamoDB Triggers (just like database triggers) is a feature which allows execution of custom actions based on item-level updates on a table
  • DynamoDB triggers can be used in scenarios like sending notifications, updating an aggregate table, and connecting DynamoDB tables to other data sources
  • DynamoDB Trigger flow
    • Custom logic for a DynamoDB trigger is stored in an AWS Lambda function as code.
    • A trigger for a given table can be created by associating an AWS Lambda function to the stream (via DynamoDB Streams) on a table.
    • When the table is updated, the updates are published to DynamoDB Streams.
    • In turn, AWS Lambda reads the updates from the associated stream and executes the code in the function.

DynamoDB Accelerator DAX

  • DynamoDB Accelerator (DAX) is a fully managed, highly available, in-memory cache for DynamoDB that delivers up to a 10x performance improvement – from milliseconds to microseconds – even at millions of requests per second.
  • DAX does all the heavy lifting required to add in-memory acceleration to the tables, without requiring developers to manage cache invalidation, data population, or cluster management.
  • DAX is fault-tolerant and scalable.
  • DAX cluster has a primary node and zero or more read-replica nodes. Upon a failure for a primary node, DAX will automatically fail over and elect a new primary. For scaling, add or remove read replicas

VPC Endpoints

  • VPC endpoints for DynamoDB improve privacy and security, especially those dealing with sensitive workloads with compliance and audit requirements, by enabling private access to DynamoDB from within a VPC without the need for an internet gateway or NAT gateway.
  • VPC endpoints for DynamoDB support IAM policies to simplify DynamoDB access control, where access can be restricted to a specific VPC endpoint.
  • VPC endpoints can be created only for Amazon DynamoDB tables in the same AWS Region as the VPC
  • DynamoDB Streams cannot be access using VPC endpoints for DynamoDB

AWS Certification Exam Practice Questions

  • Questions are collected from Internet and the answers are marked as per my knowledge and understanding (which might differ with yours).
  • AWS services are updated everyday and both the answers and questions might be outdated soon, so research accordingly.
  • AWS exam questions are not updated to keep up the pace with AWS updates, so even if the underlying feature has changed the question might not be updated
  • Open to further feedback, discussion and correction.
  1. What are the services supported by VPC endpoints, using Gateway endpoint type? Choose 2 answers
    1. Amazon S3
    2. Amazon EFS
    3. Amazon DynamoDB
    4. Amazon Glacier
    5. Amazon SQS
  2. A company has setup an application in AWS that interacts with DynamoDB. DynamoDB is currently responding in milliseconds, but the application response guidelines require it to respond within microseconds. How can the performance of DynamoDB be further improved? [SAA-C01]
    1. Use ElastiCache in front of DynamoDB
    2. Use DynamoDB inbuilt caching
    3. Use DynamoDB Accelerator
    4. Use RDS with ElastiCache instead

AWS Certified Solutions Architect – Associate SAA-C01 Exam Learning Path

AWS Certified Solutions Architect –

Associate SAA-C01 Exam Learning Path

AWS Solutions Architect – Associate SAA-C01 exam is the latest AWS exam and would replace the old CSA-Associate exam. It basically validates the ability to effectively demonstrate knowledge of how to architect and deploy secure and robust applications on AWS technologies

  • Define a solution using architectural design principles based on customer requirements.
  • Provide implementation guidance based on best practices to the organization throughout the life cycle of the project.

Refer AWS_Solution_Architect_-_Associate_SAA-C01_Exam_Blue_Print

AWS Certified Solutions Architect - Associate February 2018

AWS Solutions Architect – Associate SAA-C01 Exam Summary

  • AWS has updated the exam concepts from the focus being on individual services to more building of scalable, highly available, cost-effective, performant, resilient and operational effective architecture
  • Although, most of the services covered by the the old exam are the same. There are few new additions like API Gateway, Lambda, ECS, Aurora
  • Exam surely covers the architecture aspects in deep, so you must be able to visualize the architecture, even draw them out in the exam just to understand how it would work and how different services relate.
  • Be sure to cover the following topics
    • Networking
      • Be sure to create VPC from scratch. This is mandatory.
        • Create VPC and understand whats an CIDR.
        • Create public and private subnets, configure proper routes, security groups, NACLs.
        • Create Bastion for communication with instances
        • Create NAT Gateway or Instances for instances in private subnets to interact with internet
        • Create two tier architecture with application in public and database in private subnets
        • Create three tier architecture with web servers in public, application and database servers in private.
        • Make sure to understand how the communication happens between Internet, Public subnets, Private subnets, NAT, Bastion etc.
      • Understand VPC endpoints and what services it can help interact
      • Understand difference between NAT Gateway and NAT Instance
      • Understand how NAT high availability can be achieved
      • Understand CloudFront as CDN and the static and dynamic caching it provides, what can be its origin (it can point to on-premises sources)
      • Understand Route 53 for routing, health checks and various routing policies it provides and their use cases mainly for high availability
      • Be sure to cover ELB in deep. AWS has introduced ALB and NLB and there are lot of questions on ALB
      • Understand ALB features with its ability for content based and URL based routing with support for dynamic port mapping with ECS
    • Storage
      • Understand various storage options S3, EBS, Instance store, EFS, Glacier and what are the use cases and anti patterns for each
      • Would recommend referring Storage Options whitepaper, although a bit dated 90% still holds right
      • Understand various EBS volume types and their use cases in terms of IOPS and throughput. SSD for IOPS and HDD for throughput
      • Understand Burst performance and I/O credits to handle occasional peaks
      • Understand S3 features like different storage classes with lifecycle policies, static website hosting, versioning, Pre-Signed URLs for both upload and download, CORS
      • Understand Glacier as an archival storage with various retrieval patterns
      • Glacier Expedited retrieval now allows object retrieval within mins
      • Understand Storage gateway and its different types
    • Compute
      • Understand EC2 as a whole
      • Understand Auto Scaling and ELB, how they work together to provide High Available and Scalable solution
      • Understand EC2 various purchase types – Reserved, On-demand and Spot and their use cases
      • Understand Reserved purchase types with the introduction of Scheduled and Convertible types
      • Understand Lambda and serverless architecture, its features and use cases. How do you benefit from Lambda?
      • Understand ECS with its ability to deploy containers and micro services architecture
      • Know Elastic Beanstalk at a high level, what it provides and its ability to get an application running quickly
    • Databases
      • Understand relational and NoSQLs data storage options which include RDS, DynamoDB, Aurora and their use cases
      • Aurora has been added to the exam and most of time the questions refer to Aurora given its abilities for multiple read replicas and replication of data across AZs
      • Understand S3 is not a storage option for database
      • Understand RDS features – Read Replicas for scalability, Multi-AZ for High Availability, Automated Backups, underlying volume types
      • Understand DynamoDB with its low latency performance, DAX
      • Understand DynamoDB provisioned throughput for Read/Writes
      • Know ElastiCache use cases, mainly for caching performance
    • Analytics
      • Not much in deep, but understand what the services are and what they can do
      • Understand Redshift as a business intelligence tool
      • Know Kinesis for real time data capture and analytics
      • Atleast know what AWS Glue does, so you can eliminate the answer
    • Security
      • Understand IAM as a whole
      • Focus on IAM role and its use case especially with EC2 instance
      • Understand IAM identity providers and federation and use cases
      • Understand MFA and How would implement two factor authentication for your application
      • Understand encryption services
      • Refer Disaster Recovery whitepaper, be sure you know the different recovery types with impact on RTO/RPO.
    • Management Tools
      • Understand CloudWatch monitoring to provide operational transparency
      • Know which EC2 metrics it can track. Remember, it cannot track memory and disk space/swap utilization
      • Understand CloudWatch is extendable with custom metrics
      • Understand CloudTrail for Audit
      • Have a basic understanding of CloudFormation, OpsWorks
    • Integration Tools
      • Understand SQS as message queuing service and SNS as pub/sub notification service
      • Understand SQS features like visibility, long poll vs short poll
      • Focus on SQS as a decoupling service
      • AWS has released SQS FIFO, make sure you know the differences between standard and FIFO

NOTE: I have just marked the topics inline with the AWS Exam Blue Print. So be sure to check the same, as it is updated regularly and go through Whitepapers, FAQs and Re-Invent videos.

AWS Solutions Architect – Associate SAA-C01 Exam Resources

AWS Cloud Computing Whitepapers

AWS Solutions Architect – Associate Exam Contents

Domain 1: Design Resilient Architectures

  1. Choose reliable/resilient storage.
  2. Determine how to design decoupling mechanisms using AWS services.
  3. Determine how to design a multi-tier architecture solution.
  4. Determine how to design high availability and/or fault tolerant architectures.

Domain 2: Define Performant Architectures

  1. Choose performant storage and databases.
  2. Apply caching to improve performance.
  3. Design solutions for elasticity and scalability.

Domain 3: Specify Secure Applications and Architectures

  1. Determine how to secure application tiers.
  2. Determine how to secure data.
  3. Define the networking infrastructure for a single VPC application.

Domain 4: Design Cost-Optimized Architectures

  1. Determine how to design cost-optimized storage.
  2. Determine how to design cost-optimized compute.

Domain 5: Define Operationally-Excellent Architectures

  1. Choose design features in solutions that enable operational excellence.