AWS EC2 Spot Instances

Spot Instances

  • EC2 Spot instances allow access to spare EC2 computing capacity for up to 90% off than the On-Demand price
  • EC2 sets up the hourly price referred to as Spot price, which fluctuates depending upon the demand and supply of spot instances
  • Spot instances enable bidding on unused EC2 instances and are launched whenever the bid price exceeds the current market spot price
  • Spot Instances can be interrupted by EC2, with two minutes of notification, when EC2 needs the capacity back.
  • Spot instances are a cost-effective choice and can bring the EC2 costs down significantly
  • Spot instances can be used for applications flexible in the timing when they can run and also able to handle interruption by storing the state externally for e.g. they are well-suited for data analysis, batch jobs, background processing, and optional tasks
  • Spot instances differ from the On-Demand instances
    • they are not launched immediately
    • they can be terminated anytime
    • price varies as per the demand and supply of spot instances
  • Usual strategy involves using Spot instances with On-Demand or Reserved Instances, which provide a minimum level of guaranteed compute resources, while spot instances provide an additional computation boost
  • Spot instances can also be launched with a required duration (also known as Spot blocks), which are not interrupted due to changes in the Spot price
  • EC2 provides a data feed, sent to an S3 bucket specified during subscription, that describes the Spot instance usage and pricing
  • T2 and HS1 instance class types are not supported for Spot instances
  • Well Suited for
    • Ideal for various stateless, fault-tolerant, or flexible applications such as big data, containerized workloads, CI/CD, high-performance computing (HPC), web servers, and other test & development workloads
    • Applications that have flexible start and end times
    • Applications that are only feasible at very low compute prices
    • Users with urgent computing needs for large amounts of additional capacity

Spot Concepts

  • Spot pool – Pool of EC2 instances with the same instance type, availability zone, operating system, and network platform
  • Spot price – Current market price of a spot instance per hour as set by EC2 based on the last fulfilled bid
  • Spot bid – maximum bid price the bidder is willing to pay for the spot instance
  • Spot fleet – set of instances launched based on the criteria of the bidder
  • Spot Instance request
    • Provides the maximum price per hour that you are willing to pay for a Spot Instance. If unspecified, it defaults to the On-Demand price.
    • EC2 fulfills the request when the maximum price per hour for the request exceeds the Spot price and if capacity is available.
    • A Spot Instance request is either one-time or persistent.
    • EC2 automatically resubmits a persistent Spot request after the Spot Instance associated with the request is terminated. The Spot Instance request can optionally specify a duration for the Spot Instances.
  • Spot instance interruption – EC2 terminates the spot instances whenever the bid price is lower than the current market price or the supply has reduced. EC2 provides a Spot Instance interruption notice, which gives the instance a two-minute warning before it is interrupted.
  • EC2 Instance Rebalance Recommendation is a signal that notifies when a Spot Instance is at elevated risk of interruption. The signal provides an opportunity to proactively manage the Spot Instance in advance of the two-minute Spot Instance interruption notice
  • Bid status – provides the current state of the spot bid

Spot Instances Requests

  • Spot Instance requests must include
    • the maximum price that you’re willing to pay per hour per instance, which defaults to the On-Demand price.
    • Instance type
    • Availability Zone.
    • Desired number of instances
  • EC2 fulfills the request when the maximum price per hour for the request exceeds the Spot price and if capacity is available.
  • A Spot Instance request is either
    • One-time
      • A one-time request remains active until EC2 launches the Spot Instance, the request expires, or you cancel the request.
    • Persistent
      • EC2 automatically resubmits a persistent Spot request after the Spot Instance associated with the request is terminated.
      • A persistent Spot Instance request remains active until it expires or you cancel it, even if the request is fulfilled.
      • The Spot Instance request can optionally specify a duration for the Spot Instances.
      • Canceling spot instance requests does not terminate the instances
      • Be sure to delete the spot request before you delete the instances, else they would be launched again.

EC2 Spot Instance Requests

Spot Instances Pricing & How it works

  • EC2 sets up an hourly spot price which fluctuates depending upon the demand and supply.
  • A Spot Instance request is created by you (one-time) or EC2 (persistent) on your behalf.
  • Spot Instance requests must include
    • the maximum price that you’re willing to pay per hour per instance, which defaults to the On-Demand price.
    • other attributes like instance type and Availability Zone.
  • If the bid price exceeds the current market spot price, the request is fulfilled by Amazon till either the spot instance is terminated or the spot price increases beyond the bid price
  • Everyone pays the same market price for the period irrespective of the bid price given the bid price is more than the spot price for e.g. if the spot price is $0.20 and there are 2 bids from Customers with a bid price of $0.25 and $0.30, both customers would still pay $0.20 only
  • If the Spot instance is terminated by Amazon, you are not billed for the partial hour. However, if the spot instance is terminated by you, you will be charged for the partial hour
  • Spot instances with a predefined duration use a fixed hourly price that remains in effect for the Spot instance while it runs
  • EC2 can interrupt the Spot instance when the Spot price rises above the bid price, when the demand for Spot instances rises, or when the supply of Spot instances decreases.
  • When EC2 marks a Spot instance for termination, it provides a Spot instance termination notice, which gives the instance a two-minute warning before it terminates.
  • Termination notice warning is made available to the applications on the Spot instance using an item in the instance metadata termination-time attribute http://169.254.169.254/latest/meta-data/spot/termination-time and includes the time when the shutdown signal will be sent to the instance’s operating system
  • Relevant applications on Spot Instances should poll for the termination notice at 5-second intervals, giving it almost the entire two minutes to complete any needed processing before the instance is terminated and taken back by AWS
  • EBS-backed instance if it is a Spot instance cannot be stopped and started, but only rebooted or terminated
  • EBS-backed Spot Instance can be stopped – started, rebooted, or terminated

Pricing Example

Spot Instances Pricing Example

  • State 1 – Starting with Amazon EC2 has 5 Spot instances available
    • 6 bids available for Spot instances
    • Amazon EC2 picks up the top five priced bids and allocates a Spot instance to them
    • Spot Price is $0.10
    • Bid with the price of $0.05 is not served
  • State 2 – Supply of Amazon EC2 Spot instances reduce to 3
    • Amazon EC2 terminates the 2 spot instances with $0.10 ( the order in which the instances are terminated is determined at random )
    • Rest of the Spot instances continue
  • State 3 – New bid for Spot Instance is placed with Price $0.15 is placed
    • Spot instance with price $0.15 is fulfilled
    • Amazon EC2 terminates the single spot instances with $0.10
    • Spot Price changed to $0.15
  • State 4 New bid for Spot Instance is placed with Price $2 is placed
    • Spot instance with price $2 is fulfilled
    • Amazon EC2 terminates the single spot instances with $0.15
    • Spot Price changed to $1.00

Spot Fleets

  • Spot Fleet is a collection, or fleet, of Spot Instances, and optionally On-Demand Instances
  • Spot Fleet attempts to launch the number of Spot Instances and On-Demand Instances to meet the specified target capacity
  • Request for Spot Instances is fulfilled if there is available capacity and the maximum price specified in the request exceeds the current Spot price
  • Spot Fleet also attempts to maintain its target capacity fleet if the Spot Instances are interrupted.
  • Spot Fleet requests:
    • Request
      • Spot Fleet places an asynchronous one-time request for the desired capacity.
      • If capacity is diminished because of Spot interruptions, the fleet does not attempt to replenish Spot Instances, nor does it submit requests in alternative Spot capacity pools if capacity is unavailable.
    • Maintain
      • Spot Fleet places an asynchronous request for the desired capacity and maintains capacity by automatically replenishing any interrupted Spot Instances.
  • Spot Fleet Allocation Strategy
    • lowestPrice
      • default strategy, from the pool with the lowest price
      • cost optimization, short workload
    • diversified
      • distributed across all pools.
      • high availability, long workloads
    • capacityOptimized
      • from the pools with optimal capacity for the number of instances that are launching.
    • InstancePoolsToUseCount
      • distributed across the number of specified Spot pools that you specify.
      • Valid only when used in combination with lowestPrice.

Spot Instances Interruption

  • EC2 Instance Rebalance Recommendations and Spot Instance interruption notices can be used to gracefully handle Spot Instance interruptions.
  • EC2 Instance Rebalance Recommendation
    • is a signal that notifies when a Spot Instance is at elevated risk of interruption.
    • provides an opportunity to proactively manage the Spot Instance in advance of the two-minute Spot Instance interruption notice.
  • Spot Instance Interruption Notice
    • is a warning that is issued two minutes before EC2 interrupts a Spot Instance.
    • EC2 automatically stops or hibernates the Spot Instances on interruption, and automatically resumes the instances when we have available capacity.

Spot Instances vs On-Demand Instances

Spot Instances vs On-Demand Instances

Spot Instances Best Practices

  • Choose a reasonable bid price
    • which is low enough to suit the budget and high enough for the request to be fulfilled and should not be higher than the On-Demand bid price
  • Be flexible about instance types and Availability Zones
    • A Spot Instance pool is a set of unused EC2 instances with the same instance type (for example, m5.large) and AZ (for example, us-east-1a).
    • Be flexible about requested instance types and AZs you can deploy the workload. This gives Spot a better chance to find and allocate your required amount of compute capacity.
  • Ensure the instances are up and ready as soon as the request is fulfilled,
    • by provisioning an AMI with all the required software and load application data from user data
  • Prepare individual instances for interruptions
    • Make application fault-tolerant. Store important data regularly and externally in a place that won’t be affected by Spot instance termination for e.g., use S3, EBS, or DynamoDB.
  • Divide the work into smaller finer tasks
    • so that they can be completed and the state saved more frequently
  • Use Spot termination notice warning
    • to monitor instance status regularly
  • Use Proactive Capacity Rebalancing
    • Capacity Rebalancing helps you maintain workload availability by proactively augmenting your fleet with a new Spot Instance before a running Spot Instance receives the two-minute Spot Instance interruption notice. When Capacity Rebalancing is enabled, Auto Scaling or Spot Fleet attempts to proactively replace Spot Instances that have received a rebalance recommendation, providing the opportunity to rebalance your workload to new Spot Instances that are not at elevated risk of interruption.
    • Capacity Rebalancing complements the capacity optimized allocation strategy (which is designed to help find the most optimal spare capacity) and the mixed instances policy (which is designed to enhance availability by deploying instances across multiple instance types running in multiple Availability Zones).
  • Test applications
    • using On-Demand instances and terminating them to ensure that it handles unexpected termination gracefully

AWS Certification Exam Practice Questions

  • Questions are collected from Internet and the answers are marked as per my knowledge and understanding (which might differ with yours).
  • AWS services are updated everyday and both the answers and questions might be outdated soon, so research accordingly.
  • AWS exam questions are not updated to keep up the pace with AWS updates, so even if the underlying feature has changed the question might not be updated
  • Open to further feedback, discussion and correction.
  1. You have a video transcoding application running on Amazon EC2. Each instance polls a queue to find out which video should be transcoded, and then runs a transcoding process. If this process is interrupted, the video will be transcoded by another instance based on the queuing system. You have a large backlog of videos, which need to be transcoded, and would like to reduce this backlog by adding more instances. You will need these instances only until the backlog is reduced. Which type of Amazon EC2 instances should you use to reduce the backlog in the most cost efficient way?
    1. Reserved instances
    2. Spot instances
    3. Dedicated instances
    4. On-demand instances
  2. You have a distributed application that periodically processes large volumes of data across multiple Amazon EC2 Instances. The application is designed to recover gracefully from Amazon EC2 instance failures. You are required to accomplish this task in the most cost-effective way. Which of the following will meet your requirements?
    1. Spot Instances
    2. Reserved instances
    3. Dedicated instances
    4. On-Demand instances

References

AWS_EC2_Spot_Instances

AWS Compute Services Cheat Sheet

AWS Compute Services Cheat Sheet

Elastic Cloud Compute – EC2

  • provides scalable computing capacity
  • Features
    • Virtual computing environments, known as EC2 instances
    • Preconfigured templates for EC2 instances, known as Amazon Machine Images (AMIs), that package the bits needed for the server (including the operating system and additional software)
    • Various configurations of CPU, memory, storage, and networking capacity for your instances, known as Instance types
    • Secure login information for your instances using key pairs (public-private keys where private is kept by user)
    • Storage volumes for temporary data that’s deleted when you stop or terminate your instance, known as Instance store volumes
    • Persistent storage volumes for data using Elastic Block Store (EBS)
    • Multiple physical locations for your resources, such as instances and EBS volumes, known as Regions and Availability Zones
    • A firewall to specify the protocols, ports, and source IP ranges that can reach your instances using Security Groups
    • Static IP addresses, known as Elastic IP addresses
    • Metadata, known as tags, can be created and assigned to EC2 resources
    • Virtual networks that are logically isolated from the rest of the AWS cloud, and can optionally connect to on-premises network, known as Virtual private clouds (VPCs)

Amazon Machine Image – AMI

    • template from which EC2 instances can be launched quickly
    • does NOT span across regions, and needs to be copied
    • can be shared with other specific AWS accounts or made public

Instance Types

  • T for applications needing general usage
    • T2 instances are Burstable Performance Instances that provide a baseline level of CPU performance with the ability to burst above the baseline.
    • T2 instances accumulate CPU Credits when they are idle, and consume CPU Credits when they are active.
    • T2 Unlimited Instances can sustain high CPU performance for as long as a workload needs it at an additional cost.
  • R for applications needing more RAM or Memory
  • C for applications needing more Compute
  • M for applications needing more Medium or Moderate performance on both Memory and CPU
  • I for applications needing more IOPS
  • for applications needing more GPU

Instance Purchasing Option

  • On-Demand Instances
    • pay for instances and compute capacity that you use by the hour
    • no long-term commitments or up-front payments
  • Reserved Instances
    • provides lower hourly running costs by providing a billing discount
    • capacity reservation is applied to instances
    • suited if consistent, heavy, predictable usage
    • provides benefits with Consolidate Billing
    • can be modified to switch Availability Zones or the instance size within the same instance type, given the instance size footprint (Normalization factor) remains the same
    • pay for the entire term regardless of the usage
    • is not a physical instance that is launched, but rather a billing discount applied to the use of On-Demand Instances
  • Scheduled Reserved Instances
    • enable capacity reservations purchase that recurs on a daily, weekly, or monthly basis, with a specified start time and duration, for a one-year term.
    • Charges are incurred for the time that the instances are scheduled, even if they are not used
    • good choice for workloads that do not run continuously, but do run on a regular schedule
  • Spot Instances
    • cost-effective choice but does NOT guarantee availability
    • applications flexible in the timing when they can run and also able to handle interruption by storing the state externally
    • provides a two-minute warning if the instance is to be terminated to save any unsaved work
    • Spot blocks can also be launched with a required duration, which are not interrupted due to changes in the Spot price
    • Spot Fleet is a collection, or fleet, of Spot Instances, and optionally On-Demand Instances, which attempts to launch the number of Spot and On-Demand Instances to meet the specified target capacity
  • Dedicated Instances
    • is a tenancy option that enables instances to run in VPC on hardware that’s isolated, dedicated to a single customer
  • Dedicated Host
    • is a physical server with EC2 instance capacity fully dedicated to your use
  • Light, Medium, and Heavy Utilization Reserved Instances are no longer available for purchase and were part of the Previous Generation AWS EC2 purchasing model

Enhanced Networking

  • results in higher bandwidth, higher packet per second (PPS) performance, lower latency, consistency, scalability, and lower jitter
  • supported using Single Root – I/O Virtualization (SR-IOV) only on supported instance types
  • is supported only with a VPC (not EC2 Classic), HVM virtualization type and available by default on Amazon AMI but can be installed on other AMIs as well

Placement Group

  • Cluster Placement Group
    • provide low latency, High-Performance Computing via 10Gbps network
    • is a logical grouping on instances within a Single AZ
    • don’t span availability zones, can span multiple subnets but subnets must be in the same AZ
    • can span across peered VPCs for the same Availability Zones
    • existing instances can’t be moved into an existing placement group
    • An existing instance can be moved to a placement group, or moved from one placement group to another, or removed from a placement group, given it is in the stopped state.
    • for capacity errors, stop and start the instances in the placement group
    • use homogenous instance types which support enhanced networking and launch all the instances at once
  • Spread Placement Groups
    • is a group of instances that are each placed on distinct underlying hardware i.e. each instance on a distinct rack across AZ
    • recommended for applications that have a small number of critical instances that should be kept separate from each other.
    • reduces the risk of simultaneous failures that might occur when instances share the same underlying hardware.
  • Partition Placement Groups
    • is a group of instances spread across partitions i.e. group of instances spread across racks across AZs
    • reduces the likelihood of correlated hardware failures for the application.
    • can be used to spread deployment of large distributed and replicated workloads, such as HDFS, HBase, and Cassandra, across distinct hardware

EC2 Monitoring

  • CloudWatch provides monitoring for EC2 instances
  • Status monitoring helps quickly determine whether EC2 has detected any problems that might prevent instances from running applications.
  • Status monitoring includes
    • System Status checks – indicate issues with the underlying hardware
    • Instance Status checks – indicate issues with the underlying instance.

Elastic Load Balancer

  • Managed load balancing service and scales automatically
  • distributes incoming application traffic across multiple EC2 instances
  • is distributed system that is fault tolerant and actively monitored by AWS scales it as per the demand
  • are engineered to not be a single point of failure
  • need to Pre-Warm ELB if the demand is expected to shoot especially during load testing. AWS documentation does not mention it now.
  • supports routing traffic to instances in multiple AZs in the same region
  • performs Health Checks to route traffic only to the healthy instances
  • support Listeners with HTTP, HTTPS, SSL, TCP protocols
  • has an associated IPv4 and dual stack DNS name
  • can offload the work of encryption and decryption (SSL termination) so that the EC2 instances can focus on their main work
  • supports Cross Zone load balancing to help route traffic evenly across all EC2 instances regardless of the AZs they reside in
  • to help identify the IP address of a client
    • supports Proxy Protocol header for TCP/SSL connections
    • supports X-Forward headers for HTTP/HTTPS connections
  • supports Stick Sessions (session affinity) to bind a user’s session to a specific application instance,
    • it is not fault tolerant, if an instance is lost the information is lost
    • requires HTTP/HTTPS listener and does not work with TCP
    • requires SSL termination on ELB as it users the headers
  • supports Connection draining to help complete the in-flight requests in case an instance is deregistered
  • For High Availability, it is recommended to attach one subnet per AZ for at least two AZs, even if the instances are in a single subnet.
  • supports Static/Elastic IP (NLB only)
  • IPv4 & IPv6 support however VPC does not support IPv6. VPC now supports IPV6.
  • HTTPS listener does not support Client Side Certificate
  • For SSL termination at backend instances or support for Client Side Certificate use TCP for connections from the client to the ELB, use the SSL protocol for connections from the ELB to the back-end application, and deploy certificates on the back-end instances handling requests
  • supports a single SSL certificate, so for multiple SSL certificate multiple ELBs need to be created
  • Uses Server Name Indication to supports multiple SSL certificates

Application Load Balancer

  • supports HTTP and HTTPS (Secure HTTP) protocols
  • supports HTTP/2, which is enabled natively. Clients that support HTTP/2 can connect over TLS
  • supports WebSockets and Secure WebSockets natively
  • supports Request tracing, by default.
    • request tracing can be used to track HTTP requests from clients to targets or other services.
    • Load balancer upon receiving a request from a client, adds or updates the X-Amzn-Trace-Id header before sending the request to the target
  • supports containerized applications. Using Dynamic port mapping, ECS can select an unused port when scheduling a task and register the task with a target group using this port.
  • supports Sticky Sessions (Session Affinity) using load balancer generated cookies, to route requests from the same client to the same target
  • supports SSL termination, to decrypt the request on ALB before sending it to the underlying targets.
  • supports layer 7 specific features like X-Forwarded-For headers to help determine the actual client IP, port and protocol
  • automatically scales its request handling capacity in response to incoming application traffic.
  • supports hybrid load balancing, to route traffic to instances in VPC and an on-premises location
  • provides High Availability, by allowing more than one AZ to be specified
  • integrates with ACM to provision and bind a SSL/TLS certificate to the load balancer thereby making the entire SSL offload process very easy
  • supports multiple certificates for the same domain to a secure listener
  • supports IPv6 addressing, for an Internet facing load balancer
  • supports Cross-zone load balancing, and cannot be disabled.
  • supports Security Groups to control the traffic allowed to and from the load balancer.
  • provides Access Logs, to record all requests sent the load balancer, and store the logs in S3 for later analysis in compressed format
  • provides Delete Protection, to prevent the ALB from accidental deletion
  • supports Connection Idle Timeout – ALB maintains two connections for each request one with the Client (front end) and one with the target instance (back end). If no data has been sent or received by the time that the idle timeout period elapses, ALB closes the front-end connection
  • integrates with CloudWatch to provide metrics such as request counts, error counts, error types, and request latency
  • integrates with AWS WAF, a web application firewall that helps protect web applications from attacks by allowing rules configuration based on IP addresses, HTTP headers, and custom URI strings
  • integrates with CloudTrail to receive a history of ALB API calls made on the AWS account
  • back-end server authentication is NOT supported
  • does not provide Static, Elastic IP addresses

Network Load Balancer

  • handles volatile workloads and scale to millions of requests per second, without the need of pre-warming
  • offers extremely low latencies for latency-sensitive applications.
  • provides static IP/Elastic IP addresses for the load balancer
  • allows registering targets by IP address, including targets outside the VPC (on-premises) for the load balancer.
  • supports containerized applications. Using Dynamic port mapping, ECS can select an unused port when scheduling a task and register the task with a target group using this port.
  • monitors the health of its registered targets and routes the traffic only to healthy targets
  • enable cross-zone loading balancing only after creating the NLB
  • preserves client side source IP allowing the back-end to see client IP address. Target groups can be created with target type as instance ID or IP address. If targets registered by instance ID, the source IP addresses of the clients are preserved and provided to the applications. If register targets registered by IP address, the source IP addresses are the private IP addresses of the load balancer nodes.
  • supports both network and application target health checks.
  • supports long-lived TCP connections ideal for WebSocket type of applications
  • supports Zonal Isolation, which is designed for application architectures in a single zone and can be enabled in a single AZ to support architectures that require zonal isolation
  • does not support stick sessions

Auto Scaling

  • ensures correct number of EC2 instances are always running to handle the load by scaling up or down automatically as demand changes
  • cannot span multiple regions.
  • attempts to distribute instances evenly between the AZs that are enabled for the Auto Scaling group
  • performs checks either using EC2 status checks or can use ELB health checks to determine the health of an instance and terminates the instance if unhealthy, to launch a new instance
  • can be scaled using manual scaling, scheduled scaling or demand based scaling
  • cooldown period helps ensure instances are not launched or terminated before the previous scaling activity takes effect to allow the newly launched instances to start handling traffic and reduce load

AWS Auto Scaling & ELB

  • Auto Scaling & ELB can be used for High Availability and Redundancy by spanning Auto Scaling groups across multiple AZs within a region and then setting up ELB to distribute incoming traffic across those AZs
  • With Auto Scaling, use ELB health check with the instances to ensure that traffic is routed only to the healthy instances

AWS EC2 VM Import/Export

EC2 VM Import/Export

  • EC2 VM Import/Export enables importing virtual machine (VM) images from existing virtualization environment to EC2, and then export them back to the on-premises environment
  • EC2 VM Import/Export enables
    • migration of applications and workloads to EC2,
    • coping VM image catalog to EC2, or
    • create a repository of VM images for backup and disaster recovery
    • to leverage previous investments in building VMs by migrating the VMs to EC2.
  • Supported file formats are: VMware ESX VMDK images, Citrix Xen VHD images, Microsoft Hyper-V VHD images, and RAW images
  • For VMware vSphere, AWS Connector for vCenter can be used to export a VM from VMware and import it into Amazon EC2
  • For Microsoft Systems Center, AWS Systems Manager for Microsoft SCVMM can be used to import Windows VMs from SCVMM to EC2

AWS EC2 VM Import/Export

EC2 VM Import/Export features

  • ability to import a VM from a virtualization environment to EC2 as an Amazon Machine Image (AMI), which can be used to launch an EC2 instance
  • ability to import a VM from a virtualization environment to EC2 as an EC2 instance, which is initially in a stopped state. AMI can be created from it
  • ability to export a VM that was previously imported from the virtualization environment
  • ability to import disks as EBS snapshots.

AWS Certification Exam Practice Questions

  • Questions are collected from Internet and the answers are marked as per my knowledge and understanding (which might differ with yours).
  • AWS services are updated everyday and both the answers and questions might be outdated soon, so research accordingly.
  • AWS exam questions are not updated to keep up the pace with AWS updates, so even if the underlying feature has changed the question might not be updated
  • Open to further feedback, discussion and correction.
  1. You are responsible for a legacy web application whose server environment is approaching end of life. You would like to migrate this application to AWS as quickly as possible, since the application environment currently has the following limitations: The VM’s single 10GB VMDK is almost full. The virtual network interface still uses the 10Mbps driver, which leaves your 100Mbps WAN connection completely underutilized. It is currently running on a highly customized Windows VM within a VMware environment: You do not have the installation media. This is a mission critical application with an RTO (Recovery Time Objective) of 8 hours. RPO (Recovery Point Objective) of 1 hour. How could you best migrate this application to AWS while meeting your business continuity requirements?
    1. Use the EC2 VM Import Connector for vCenter to import the VM into EC2
    2. Use Import/Export to import the VM as an EBS snapshot and attach to EC2. (Import/Export is used to transfer large amount of data)
    3. Use S3 to create a backup of the VM and restore the data into EC2.
    4. Use the ec2-bundle-instance API to Import an Image of the VM into EC2 (only bundles an windows instance store instance)
  2. You are tasked with moving a legacy application from a virtual machine running inside your datacenter to an Amazon VPC. Unfortunately this app requires access to a number of on-premises services and no one who configured the app still works for your company. Even worse there’s no documentation for it. What will allow the application running inside the VPC to reach back and access its internal dependencies without being reconfigured? (Choose 3 answers)
    1. An AWS Direct Connect link between the VPC and the network housing the internal services (VPN or a DX for communication)
    2. An Internet Gateway to allow a VPN connection. (Virtual and Customer gateway is needed)
    3. An Elastic IP address on the VPC instance (Don’t need a EIP as private subnets can also interact with on-premises network)
    4. An IP address space that does not conflict with the one on-premises (IP address cannot conflict)
    5. Entries in Amazon Route 53 that allow the Instance to resolve its dependencies’ IP addresses (Route 53 is not required)
    6. A VM Import of the current virtual machine (VM Import to copy the VM to AWS as there is no documentation it can’t be configured from scratch)

AWS EC2 Network – Enhanced Networking

EC2 Enhanced Networking

  • Enhanced networking results in higher bandwidth, higher packet per second (PPS) performance, lower latency, consistency, scalability and lower jitter
  • EC2 provides enhanced networking capabilities using single root I/O virtualization (SR-IOV) only on supported instance types
    • SR-IOV is a method of device virtualization that provides higher I/O performance and lower CPU utilization
  • Amazon Linux AMIs and Windows Server 2012 R2 AMI already have the module installed with the attributes set and do not require any additional configurations.
  • It can be enabled for other OS distributions by installing the module with the correct attributes configured
  • Enhanced Networking is supported using
    • Elastic Network Adapter (ENA)
      • The Elastic Network Adapter (ENA) supports network speeds of up to 100 Gbps for supported instance types.
      • The current generation instances use ENA for enhanced networking, except for C4, D2, and M4 instances smaller than m4.16xlarge.
    • Intel 82599 Virtual Function (VF) interface
      • The Intel 82599 Virtual Function interface supports network speeds of up to 10 Gbps for supported instance types.
      • supported instance types: C3, C4, D2, I2, M4 (excl. m4.16xlarge), and R3.

VF Enhanced Networking Key Requirements

  • VPC, as enhanced networking can’t be enabled for instance in EC2-Classic
  • an HVM virtualization type AMI
  • Instance kernel version
    • Linux kernel version of 2.6.32+
    • Windows: Server 2008 R2+
  • Appropriate Virtual Function (VF) driver
    • Linux – should have the ixgbevf module installed and that sriovNetSupport attribute set for the instance
    • Windows- Intel 82599 Virtual Function driver
  • supported instance types: C3, C4, D2, I2, M4 (excl. m4.16xlarge), and R3.

AWS Certification Exam Practice Questions

  • Questions are collected from Internet and the answers are marked as per my knowledge and understanding (which might differ with yours).
  • AWS services are updated everyday and both the answers and questions might be outdated soon, so research accordingly.
  • AWS exam questions are not updated to keep up the pace with AWS updates, so even if the underlying feature has changed the question might not be updated
  • Open to further feedback, discussion and correction.
  1. You have multiple Amazon EC2 instances running in a cluster across multiple Availability Zones within the same region. What combination of the following should be used to ensure the highest network performance (packets per second), lowest latency, and lowest jitter? Choose 3 answers
    1. Amazon EC2 placement groups (would not work for multiple AZs)
    2. Enhanced networking (provides network performance, lowest latency)
    3. Amazon PV AMI (Requires HVM)
    4. Amazon HVM AMI (Requires HVM)
    5. Amazon Linux (Can be on others as well)
    6. Amazon VPC (works only in VPC, can’t enable enhanced networking if the instance is in EC2-Classic)
  2. A group of researchers is studying the migration pattern of a beetle that eats and destroys gram. The researchers must process massive amounts of data and run statistics. Which one of the following options provides the high performance computing for this purpose.
    1. Configure an Autoscaling Scaling group to launch dozens of spot instances to run the statistical analysis simultaneously
    2. Launch AMI instances that support SR-IOV in a single Availability Zone
    3. Launch compute optimized (C4) instances in at least two Availability Zones
    4. Launch enhanced network type instances in a placement group

References

AWS EC2 Monitoring

EC2 Monitoring

Status Checks

  • Status monitoring helps quickly determine whether EC2 has detected any problems that might prevent instances from running applications.
  • EC2 performs automated checks on every running EC2 instance to identify hardware and software issues.
  • Status checks are performed every minute and each returns a pass or a fail status.
  • If all checks pass, the overall status of the instance is OK.
  • If one or more checks fail, the overall status is Impaired.
  • Status checks are built into EC2, so they cannot be disabled or deleted.
  • Status checks data augments the information that EC2 already provides about the intended state of each instance (such as pending, running, stopping) as well as the utilization metrics that CloudWatch monitors (CPU utilization, network traffic, and disk activity).
  • Alarms can be created or deleted, that are triggered based on the result of the status checks. for e.g., an alarm can be created to warn if status checks fail on a specific instance.

System Status Checks

  • monitor the AWS systems, required to use the instance, to ensure they are working properly.
  • detect problems with the instance that require AWS involvement to repair.
  • System status checks failure might due to
    • Loss of network connectivity
    • Loss of system power
    • Software issues on the physical host
    • Hardware issues on the physical host
  • When a system status check fails, one can either
    • check Personal Health Dashboard for any scheduled critical maintenance by AWS to the instance’s host.
    • wait for AWS to fix the issue
    • or resolve it by stopping and restarting or terminating and replacing an instance

Instance Status Checks

  • monitor the software and network configuration of the individual instance
  • checks to detect problems that require involvement to repair.
  • Instance status checks failure might be due to
    • Failed system status checks
    • Misconfigured networking or startup configuration
    • Exhausted memory
    • Corrupted file system
    • Incompatible kernel
  • When an instance status check fails, it can be resolved by either rebooting the instance or by making modifications in the operating system

CloudWatch Monitoring

  • CloudWatch helps monitor EC2 instances, which collects and processes
    raw data from EC2 into readable, near real-time metrics.
  • Statistics are recorded for a period of two weeks so that historical information can be accessed and used to gain a better perspective on how
    the application or service is performing.
  • By default, Basic monitoring is enabled and EC2 metric data is sent to CloudWatch in 5-minute periods automatically
  • Detailed monitoring can be enabled on the EC2 instance, which sends data to CloudWatch in 1-minute periods.
  • Aggregating Statistics Across Instances/ASG/AMI ID
    • Aggregate statistics are available for the instances that have detailed monitoring (at an additional charge) enable, which provides data in 1-minute periods
    • Instances that use basic monitoring are not included in the aggregates.
    • CloudWatch does not aggregate data across Regions. Therefore, metrics are completely separate between regions.
    • CloudWatch returns statistics for all dimensions in the AWS/EC2 namespace if no dimension is specified
    • The technique for retrieving all dimensions across an AWS namespace does not work for custom namespaces published to CloudWatch.
    • Statistics include Sum, Average, Minimum, Maximum, Data Samples
    • With custom namespaces, the complete set of dimensions that are associated with any given data point to retrieve statistics that include the data point must be specified
  • CloudWatch alarms
    • can be created to monitor any one of the EC2 instance’s metrics.
    • can be configured to automatically send you a notification when the metric reaches a specified threshold.
    • can automatically stop, terminate, reboot, or recover EC2 instances
    • can automatically recover an EC2 instance when the instance becomes impaired due to an underlying hardware failure a problem that requires AWS involvement to repair
    • can automatically stop or terminate the instances to save costs (EC2 instances that use an EBS volume as the root device can be stopped
      or terminated, whereas instances that use the instance store as the root device can only be terminated)
    • can use EC2ActionsAccess IAM role, which enables AWS to perform stop, terminate, or reboot actions on EC2 instances
    • If you have read/write permissions for CloudWatch but not for EC2, alarms can still be created but the stop or terminate actions won’t be performed on the EC2 instance

EC2 Monitoring Metrics

  • CPUCreditUsage
    • (Only valid for T2 instances) The number of CPU credits consumed
      during the specified period.
    • This metric identifies the amount of time during which physical CPUs
      were used for processing instructions by virtual CPUs allocated to
      the instance.
    • CPU Credit metrics are available at a 5-minute frequency.
  • CPUCreditBalance
    • (Only valid for T2 instances) The number of CPU credits that an instance has accumulated.
    • This metric is used to determine how long an instance can burst beyond its baseline performance level at a given rate.
    • CPU Credit metrics are available at a 5-minute frequency.
  • CPUUtilization
    • % of allocated EC2 compute units that are currently in use on the instance. This metric identifies the processing power required to run an application upon a selected instance.
  • DiskReadOps
    • Completed read operations from all instance store volumes available to the instance in a specified period of time.
  • DiskWriteOps
    • Completed write operations to all instance store volumes available to the instance in a specified period of time.
  • DiskReadBytes
    • Bytes read from all instance store volumes available to the instance.
    • This metric is used to determine the volume of the data the application reads from the hard disk of the instance.
    • This can be used to determine the speed of the application.
  • DiskWriteBytes
    • Bytes written to all instance store volumes available to the instance.
    • This metric is used to determine the volume of the data the application writes onto the hard disk of the instance.
    • This can be used to determine the speed of the application.
  • NetworkIn
    • The number of bytes received on all network interfaces by the instance. This metric identifies the volume of incoming network traffic to an application on a single instance.
  • NetworkOut
    • The number of bytes sent out on all network interfaces by the instance. This metric identifies the volume of outgoing network traffic to an application on a single instance.
  • NetworkPacketsIn
    • The number of packets received on all network interfaces by the instance. This metric identifies the volume of incoming traffic in terms of the number of packets on a single instance.
    • This metric is available for basic monitoring only
  • NetworkPacketsOut
    • The number of packets sent out on all network interfaces by the instance. This metric identifies the volume of outgoing traffic in terms of the number of packets on a single instance.
    • This metric is available for basic monitoring only.
  • StatusCheckFailed
    • Reports if either of the status checks, StatusCheckFailed_Instance and StatusCheckFailed_System that has failed.
    • Values for this metric are either 0 (zero) or 1 (one.) A zero indicates that the status checks passed. A one indicates a status check failure.
    • Status check metrics are available at a 1-minute frequency
  • StatusCheckFailed_Instance
    • Reports whether the instance has passed the EC2 instance status check in the last minute.
    • Values for this metric are either 0 (zero) or 1 (one.) A zero indicates that the status checks passed. A one indicates a status check failure.
    • Status check metrics are available at a 1-minute frequency
  • StatusCheckFailed_System
    • Reports whether the instance has passed the EC2 system status check in the last minute.
    • Values for this metric are either 0 (zero) or 1 (one.) A zero indicates that the status checks passed. A one indicates a status check failure.
    • Status check metrics are available at a 1-minute frequency

AWS Certification Exam Practice Questions

  • Questions are collected from Internet and the answers are marked as per my knowledge and understanding (which might differ with yours).
  • AWS services are updated everyday and both the answers and questions might be outdated soon, so research accordingly.
  • AWS exam questions are not updated to keep up the pace with AWS updates, so even if the underlying feature has changed the question might not be updated
  • Open to further feedback, discussion and correction.
  1. In the basic monitoring package for EC2, Amazon CloudWatch provides the following metrics:
    1. Web server visible metrics such as number failed transaction requests
    2. Operating system visible metrics such as memory utilization
    3. Database visible metrics such as number of connections
    4. Hypervisor visible metrics such as CPU utilization
  2. Which of the following requires a custom CloudWatch metric to monitor?
    1. Memory Utilization of an EC2 instance
    2. CPU Utilization of an EC2 instance
    3. Disk usage activity of an EC2 instance
    4. Data transfer of an EC2 instance
  3. A user has configured CloudWatch monitoring on an EBS backed EC2 instance. If the user has not attached any additional device, which of the below mentioned metrics will always show a 0 value?
    1. DiskReadBytes
    2. NetworkIn
    3. NetworkOut
    4. CPUUtilization
  4. A user is running a batch process on EBS backed EC2 instances. The batch process starts a few instances to process Hadoop Map reduce jobs, which can run between 50 – 600 minutes or sometimes for more time. The user wants to configure that the instance gets terminated only when the process is completed. How can the user configure this with CloudWatch?
    1. Setup the CloudWatch action to terminate the instance when the CPU utilization is less than 5%
    2. Setup the CloudWatch with Auto Scaling to terminate all the instances
    3. Setup a job which terminates all instances after 600 minutes
    4. It is not possible to terminate instances automatically
  5. An AWS account owner has setup multiple IAM users. One IAM user only has CloudWatch access. He has setup the alarm action, which stops the EC2 instances when the CPU utilization is below the threshold limit. What will happen in this case?
    1. It is not possible to stop the instance using the CloudWatch alarm
    2. CloudWatch will stop the instance when the action is executed
    3. The user cannot set an alarm on EC2 since he does not have the permission
    4. The user can setup the action but it will not be executed if the user does not have EC2 rights
  6. A user has launched 10 instances from the same AMI ID using Auto Scaling. The user is trying to see the average CPU utilization across all instances of the last 2 weeks under the CloudWatch console. How can the user achieve this?
    1. View the Auto Scaling CPU metrics (Refer AS Instance Monitoring)
    2. Aggregate the data over the instance AMI ID (Works but needs detailed monitoring enabled)
    3. The user has to use the CloudWatchanalyser to find the average data across instances
    4. It is not possible to see the average CPU utilization of the same AMI ID since the instance ID is different

References

EC2_Monitoring

AWS EC2 Troubleshooting

AWS EC2 Troubleshooting

An Instance Immediately Terminates

  • EBS volume limit was reached. Its a soft limit and can be increased by submitting a support request
  • EBS snapshot is corrupt.
  • Root EBS volume is encrypted and you do not have permission to access the KMS key for decryption.
  • Instance store-backed AMI used to launch the instance is missing a required part
  • Resolution
    • delete unused volumes
    • ensure proper permissions to access the AWS keys.

Connecting to the Instance Issues

  • Error connecting to your instance: Connection timed out
    • Route table, for the subnet, does not have a  route that sends all traffic destined outside the VPC to the Internet gateway for the VPC.
    • Security group does not allow inbound traffic from the public IP address on the proper port
    • ACL does not allow inbound traffic from and outbound traffic to the public IP address on the proper port
    • Private key used to connect does not match with key that corresponds to the key pair selected for the instance during the launch
    • Appropriate user name for the AMI is not used for e.g. user name for Amazon Linux AMI is ec2-user, Ubuntu AMI is ubuntu, RHEL5 AMI & SUSE Linux can be either root or ec2-user, Fedora AMI can be fedora or ec2-user
    • If connecting from a corporate network, the internal firewall does not
      allow inbound and outbound traffic on port 22 (for Linux instances) or port 3389 (for Windows instances).
    • Instance does not have the same public IP address, which changes during restarts. Associate an Elastic IP address with the instance
    • CPU load on the instance is high; the server may be overloaded.
  • User key not recognized by the server
    • private key file used to connect has not been converted to the format as required by the server
  • Host key not found, Permission denied (publickey), or Authentication failed, permission denied
    • appropriate user name for the AMI is not used for connecting
    • proper private key file for the instance is not used
  • Unprotected Private Key File
    • private key file is not protected from read and write operations from any other users.
  • Server refused our key or No supported authentication methods available
    • appropriate user name for the AMI is not used for connecting

Failed Status Checks

  • System Status Check – Checks Physical Hosts
    • Lost of Network connectivity
    • Loss of System power
    • Software issues on physical host
    • Hardware issues on physical host
    • Resolution
      • Amazon EBS-backed AMI instance – stop and restart the instance
      • Instance-store backed AMI – terminate the instance and launch a replacement.
  • Instance Status Check – Checks Instance or VM
    • Possible reasons
      • Misconfigured networking or startup configuration
      • Exhausted memory
      • Corrupted file system
      • Failed Amazon EBS volume or Physical drive
      • Incompatible kernel
    • Resolution
      • Rebooting of the Instance or making modifications in your Operating system, volumes

Instance Capacity Issues

  • InsufficientInstanceCapacity
    • AWS does not currently have enough available capacity to service the request.
    • There is a limit to the number of instances of instance type that can be launched within a region.
    • Issue is mainly from the AWS side and it can be resolved by
      • reducing the request for the number of instances
      • changing the instance type
      • submiting a request without specifying the Availability Zone.
  • InstanceLimitExceeded
    • Concurrent running instance limit, default is 20, has been reached in a region.
    • Request an instance limit increase on a per-region basis

AWS Certification Exam Practice Questions

  • Questions are collected from Internet and the answers are marked as per my knowledge and understanding (which might differ with yours).
  • AWS services are updated everyday and both the answers and questions might be outdated soon, so research accordingly.
  • AWS exam questions are not updated to keep up the pace with AWS updates, so even if the underlying feature has changed the question might not be updated
  • Open to further feedback, discussion and correction.
  1. A user has launched an EC2 instance. The instance got terminated as soon as it was launched. Which of the below mentioned options is not a possible reason for this?
    1. The user account has reached the maximum EC2 instance limit (Refer link)
    2. The snapshot is corrupt
    3. The AMI is missing. It is the required part
    4. The user account has reached the maximum volume limit
  2. If you’re unable to connect via SSH to your EC2 instance, which of the following should you check and possibly correct to restore connectivity?
    1. Adjust Security Group to permit egress traffic over TCP port 443 from your IP.
    2. Configure the IAM role to permit changes to security group settings.
    3. Modify the instance security group to allow ingress of ICMP packets from your IP.
    4. Adjust the instance’s Security Group to permit ingress traffic over port 22 from your IP
    5. Apply the most recently released Operating System security patches.
  3. You try to connect via SSH to a newly created Amazon EC2 instance and get one of the following error messages: “Network error: Connection timed out” or “Error connecting to [instance], reason: -> Connection timed out: connect,” You have confirmed that the network and security group rules are configured correctly and the instance is passing status checks. What steps should you take to identify the source of the behavior? Choose 2 answers
    1. Verify that the private key file corresponds to the Amazon EC2 key pair assigned at launch.
    2. Verify that your IAM user policy has permission to launch Amazon EC2 instances. (there is not need for a IAM user and just need ssh keys)
    3. Verify that you are connecting with the appropriate user name for your AMI. (Although it gives different error seems the only other logical choice)
    4. Verify that the Amazon EC2 Instance was launched with the proper IAM role. (role assigned to EC2 is irrelevant for ssh and only controls what AWS resources EC2 can access to)
    5. Verify that your federation trust to AWS has been established (federation is for authenticating the user)
  4. A user has launched an EBS backed EC2 instance in the us-east-1a region. The user stopped the instance and started it back after 20 days. AWS throws up an ‘Insufficient Instance Capacity’ error. What can be the possible reason for this?
    1. AWS does not have sufficient capacity in that availability zone
    2. AWS zone mapping is changed for that user account
    3. There is some issue with the host capacity on which the instance is launched
    4. The user account has reached the maximum EC2 instance limit
  5. A user is trying to connect to a running EC2 instance using SSH. However, the user gets an Unprotected Private Key File error. Which of the below mentioned options can be a possible reason for rejection?
    1. The private key file has the wrong file permission
    2. The ppk file used for SSH is read only
    3. The public key file has the wrong permission
    4. The user has provided the wrong user name for the OS login
  6. A user has launched an EC2 instance. However, due to some reason the instance was terminated. If the user wants to find out the reason for termination, where can he find the details?
    1. It is not possible to find the details after the instance is terminated
    2. The user can get information from the AWS console, by checking the Instance description under the State transition reason label
    3. The user can get information from the AWS console, by checking the Instance description under the Instance Status Change reason label
    4. The user can get information from the AWS console, by checking the Instance description under the Instance Termination reason label
  7. You have a Linux EC2 web server instance running inside a VPC. The instance is in a public subnet and has an EIP associated with it so you can connect to it over the Internet via HTTP or SSH. The instance was also fully accessible when you last logged in via SSH and was also serving web requests on port 80. Now you are not able to SSH into the host nor does it respond to web requests on port 80, that were working fine last time you checked. You have double-checked that all networking configuration parameters (security groups route tables, IGW, EIP. NACLs etc.) are properly configured and you haven’t made any changes to those anyway since you were last able to reach the Instance). You look at the EC2 console and notice that system status check shows “impaired.” Which should be your next step in troubleshooting and attempting to get the instance back to a healthy state so that you can log in again?
    1. Stop and start the instance so that it will be able to be redeployed on a healthy host system that most likely will fix the “impaired” system status (for system status check impaired status you need Stop Start for EBS and terminate and relaunch for Instance store)
    2. Reboot your instance so that the operating system will have a chance to boot in a clean healthy state that most likely will fix the ‘impaired” system status
    3. Add another dynamic private IP address to me instance and try to connect via that new path, since the networking stack of the OS may be locked up causing the “impaired” system status.
    4. Add another Elastic Network Interface to the instance and try to connect via that new path since the networking stack of the OS may be locked up causing the “impaired” system status
    5. un-map and then re-map the EIP to the instance, since the IGW/NAT gateway may not be working properly, causing the “impaired” system status
  8. A user is trying to connect to a running EC2 instance using SSH. However, the user gets a connection time out error. Which of the below mentioned options is not a possible reason for rejection?
    1. The access key to connect to the instance is wrong (access key is different from ssh private key)
    2. The security group is not configured properly
    3. The private key used to launch the instance is not correct
    4. The instance CPU is heavily loaded
  9. A user is trying to connect to a running EC2 instance using SSH. However, the user gets a Host key not found error. Which of the below mentioned options is a possible reason for rejection?
    1. The user has provided the wrong user name for the OS login
    2. The instance CPU is heavily loaded
    3. The security group is not configured properly
    4. The access key to connect to the instance is wrong (access key is different from ssh private key)

AWS EC2 Network Features

EC2 Network Features

EC2 Network covers a lot of features for low latency access, High Performance Computing, Enhanced Networking, etc.

EC2 and VPC

  • All the EC2 instance types can be launched in a VPC
  • Instance types C4, M4 & T2 are available in VPC only and cannot be launched in EC2-Classic
  • Launching an EC2 instance within a VPC provides the following benefits
    • Assign static private IP addresses to instances that persist across starts and stops
    • Assign multiple IP addresses to the instances
    • Define network interfaces, and attach one or more network interfaces to the instances
    • Change security group membership for the instances while they’re running
    • Control the outbound traffic from the instances (egress filtering) in addition to controlling the inbound traffic to them (ingress filtering)
    • Add an additional layer of access control to the instances in the form of network access control lists (ACL)
    • Run the instances on single-tenant dedicated hardware

EC2 Instance IP Addressing

  • Private IP address & Internal DNS Hostnames
    • Private IP address is the IP address that’s not reachable over the internet and can be resolved only within the network
    • When an instance is launched, the default network interface eth0 is assigned a private IP address and an internal DNS hostname, which resolves to the private IP address and can be used for communication between the instances in the same network only
    • Private IP address and DNS hostname cannot be resolved outside the network that the instance is in
    • Private IP address behaviour
      • remains associated with the instance when it is stopped or rebooted
      • is disassociated only when the instance is terminated
    • An instance when launched can be assigned a private IP address or EC2 will automatically assign an IP address to the instance within the address range of the subnet
    • An additional private IP addresses, known as secondary private IP addresses can also be assigned. Unlike primary private IP addresses, secondary private IP addresses can be reassigned from one instance to another.
  • Public IP address and External DNS hostnames
    • A public IP address is reachable from the Internet
    • Each instance assigned a public IP address is also given an External DNS hostname.
    • External DNS hostname resolves to the public IP address outside the network and to the private IP address within the network.
    • Public IP address is associated with the primary Private IP address through NAT
    • Within a VPC, an instance may or may not be assigned a public IP address depending upon the subnet Assign Public IP attribute
    • Public IP address assigned to the pool is from the public IP address pool and is assigned to the instance, and not to the AWS account. It cannot be reused once disassociated and is released back to the pool
    • Public IP address behaviour
      • cannot be manually associated or disassociated with an instance
      • is released when an instance is stopped or terminated.
      • a new public IP address is assigned when stopped instance is started
      • is released when an instance is assigned an Elastic IP address
      • is not assigned if there are more than one network interface attached to the instance
  • Multiple Private IP addresses
    • In EC2-VPC, multiple private IP addresses can be specified to the instances.
    • This can be useful in the following cases
      • Host multiple websites on a single server by using multiple SSL certificates on a single server and associating each certificate with a specific IP address.
      • Operate network appliances, such as firewalls or load balancers, that have multiple private IP addresses for each network interface.
      • Redirect internal traffic to a standby instance in case the instance fails, by reassigning the secondary private IP address to the standby instance.
    • Multiple IP addresses work with Network Interfaces
      • Secondary IP address can be assigned to any network interface, which can be attached or detached from an instance
      • Secondary IP address must be assigned from the CIDR block range of the subnet for the network interface
      • Security groups apply to network interfaces and not to IP addresses
      • Secondary private IP addresses can be assigned and unassigned to ENIs attached to running or stopped instances.
      • Secondary private IP addresses that are assigned to a network interface can be reassigned to another one if you explicitly allow it.
      • Primary private IP addresses, secondary private IP addresses, and any associated Elastic IP addresses remain with the network interface when it is detached from an instance or attached to another instance.
      • Although primary network interface cannot be moved from an instance, the secondary private IP address of the primary network interface can be reassigned to another network interface.

Elastic IP Addresses

  • An Elastic IP address is a static IP address designed for dynamic cloud computing.
  • Elastic IP address can help mask the failure of an instance or software by rapidly remapping the address to another instance in the account.
  • Elastic IP address is associated with the AWS account and it remains associated with the account until released explicitly
  • Elastic IP address is NOT associated with a particular instance
  • When an instance is launched in the default VPC, it is assigned 2 IP addresses, a private and a public IP address, which is mapped to the private IP address through NAT
  • An instance launched in a non-default VPC is assigned only a private IP address unless a public address is specifically requested or the subnet public IP attribute is enabled
  • When an Elastic IP address is assigned to an instance, the public IP address is disassociated with the instance
  • For an instance, without a public IP address, to communicate to the internet it must be assigned an Elastic IP address
  • When the Elastic IP address is dissociated the public IP address is assigned back to the instance. However, if a secondary network interface is attached to the instance, the public IP address is not automatically assigned
  • Elastic IP addresses are not charged when associated with a running instance
  • Amazon imposes a small hourly fee for an unused Elastic IP address to ensure efficient use of Elastic IP addresses. So charges would be applied, if it is not associated or associated with an instance in a stopped state or associated with an unattached network interface.
  • All AWS accounts are limited to 5 EIPs (soft limit) because public (IPv4) Internet addresses are a scarce public resource

EC2 Classic, Default & Non Default Subnet Comparision

Elastic Network Interfaces (ENI)

  • Elastic Network Interfaces (ENIs) are virtual network interfaces that can be attached to the instances running in an VPC only
  • ENI consists of the following
    • A primary private IP address.
    • One or more secondary private IP addresses
    • One Elastic IP address per private IP address.
    • One public IP address, which can be auto-assigned to the elastic network interface for eth0 when an instance is launched, but only when elastic network interface for eth0 is created instead of using an existing network interface
    • One or more security groups
    • A MAC address
    • A source/destination check flag
    • A description
  • ENI can be created without being attached to an instance
  • ENI can be attached to an instance, detached from that instance and attached to an other instance. Attributes of an ENI like elastic IP address, private IP address follow the ENI and when moved from one instance to an other instance & all traffic to the ENI will be routed to the new instance.
  • An instance in VPC always has a default primary ENI attached (eth0) with a private ip address assigned from VPC range and cannot be detached
  • Additional ENI (eth1-ethn) can be attached to the instance and the number varies depending upon the instance type
  • Most important difference between eth0 and eth1 is that eth0 cannot be dynamically attached or detached from a running instance.
  • Primary ENIs (eth0) are created automatically when an EC2 instance is launched and are also deleted automatically when the instance is terminated unless the administrator has changed a property of the ENI to keep it alive afterward
  • Multiple elastic network interfaces are useful for use cases:
    • Create a management network
      • Primary ENI eth0 handles backend with more restrictive control
      • Secondary ENI eth1 handles the public facing traffic
    • Licensing authentication
      • Fixed MAC address associated with a license authentication
    • Use network and security appliances in your VPC
      • configure a third party network and security appliances (load balancers, NAT, proxy) with the secondary ENI
    • Create dual-homed instances with workloads/roles on distinct subnets.
    • Create a low-budget, high-availability solution
      • If one of the instances serving a particular function fails, its elastic network interface can be attached to a replacement or hot standby instance pre-configured for the same role in order to rapidly recover the service
      • As the interface maintains its private IP, EIP, and MAC address, network traffic will begin flowing to the standby instance as soon as it is attached to the replacement instance
  • ENI Best Practices
    • ENI can be attached to an instance when it’s running (hot attach), when it’s stopped (warm attach), or when the instance is being launched (cold attach).
    • Primary (eth0) interface can’t be detached
    • Secondary (ethN) ENI can be detached when the instance is running or stopped.
    • ENI in one subnet can be attached to an instance in another subnet, but the same AZ and same VPC
  • When launching an instance from the CLI or API, both the primary (eth0) and additional elastic network interfaces can be specified
  • Launching an Amazon Linux or Microsoft Windows Server instance with multiple network interfaces automatically configures interfaces, private IP addresses, and route tables on the operating system of the instance.
  • A warm or hot attach of an additional ENI may require bringing up the second interface manually, configure the private IP address, and modify the route table accordingly.
  • Instances running Amazon Linux or Microsoft Windows Server automatically recognize the warm or hot attach and configure themselves.
  • Attaching another ENI to an instance is not a method to increase or double the network bandwidth to or from the dual-homed instance.

Placement Groups

Refer Blog Post @ Placement Groups

Network Maximum Transmission Unit – MTU

  • MTU of a network connection is the size, in bytes, of the largest permissible packet that can be transferred over the connection.
  • The larger the MTU of the connection the more the data can be transferred in a single packet
  • Largest ethernet packet size supported over most of the internet is 1500 MTU
  • Jumbo Frames
    • Jumbo frames are Ethernet frames that allow more than 1500 bytes of data by increasing the payload size per packet, and thus increasing the percentage of the packet that is not packet overhead.
    • Fewer packets are needed to send the same amount of usable data
    • Jumbo frames should be used with caution for Internet-bound traffic or any traffic that leaves a VPC.
    • Packets are fragmented by intermediate systems, which slows down this traffic.
  • Maximum supported MTU for an instance depends on its instance type
  • All EC2 instance types support 1500 MTU, and many current instance sizes support 9001 MTU or Jumbo frames
  • Traffic is limited to a maximum MTU of 1500 in the following cases:
    • Traffic outside of a given AWS Region for EC2-Classic
    • Traffic outside of a single VPC
    • Traffic over an inter-region VPC peering connection
    • Traffic over VPN connections
    • Traffic over an internet gateway
  • For instances that are collocated inside a placement group, jumbo frames help to achieve the maximum network throughput possible, and they are recommended in this case

Enhanced Networking

Refer Blog Post @ EC2 Enhanced Networking

Elastic Fabric Adapter – EFA

  • An Elastic Fabric Adapter (EFA) is a network device that can be attached to the EC2 instance to accelerate High Performance Computing (HPC) and machine learning applications.
  • EFA helps achieve the application performance of an on-premises HPC cluster, with the scalability, flexibility, and elasticity provided by AWS.
  • EFA provides lower and more consistent latency and higher throughput than the TCP transport traditionally used in cloud-based HPC systems.
  • EFA enhances the performance of inter-instance communication that is critical for scaling HPC and machine learning applications.
  • EFA is optimized to work on the existing AWS network infrastructure and it can scale depending on application requirements.

AWS Certification Exam Practice Questions

  • Questions are collected from Internet and the answers are marked as per my knowledge and understanding (which might differ with yours).
  • AWS services are updated everyday and both the answers and questions might be outdated soon, so research accordingly.
  • AWS exam questions are not updated to keep up the pace with AWS updates, so even if the underlying feature has changed the question might not be updated
  • Open to further feedback, discussion and correction.
  1. A user is launching an EC2 instance in the US East region. Which of the below mentioned options is recommended by AWS with respect to the selection of the availability zone?
    1. Always select the US-East-1-a zone for HA
    2. Do not select the AZ; instead let AWS select the AZ
    3. The user can never select the availability zone while launching an instance
    4. Always select the AZ while launching an instance
  2. You have multiple Amazon EC2 instances running in a cluster across multiple Availability Zones within the same region. What combination of the following should be used to ensure the highest network performance (packets per second), lowest latency, and lowest jitter? Choose 3 answers
    1. Amazon EC2 placement groups (would not work for multiple AZs. Defaults to Cluster)
    2. Enhanced networking (provides network performance, lowest latency)
    3. Amazon PV AMI (Needs HVM)
    4. Amazon HVM AMI
    5. Amazon Linux (Can work on other flavors of Unix as well)
    6. Amazon VPC (Enhanced networking works only in VPC)
  3. Regarding the attaching of ENI to an instance, what does ‘warm attach’ refer to?
    1. Attaching an ENI to an instance when it is stopped
    2. Attaching an ENI to an instance when it is running
    3. Attaching an ENI to an instance during the launch process
  4. Can I detach the primary (eth0) network interface when the instance is running or stopped?
    1. Yes, You can.
    2. You cannot
    3. Depends on the state of the interface at the time
  5. By default what are ENIs that are automatically created and attached to instances using the EC2 console set to do when the attached instance terminates?
    1. Remain as is
    2. Terminate
    3. Hibernate
    4. Pause
  6. Select the incorrect statement
    1. In Amazon EC2, the private IP addresses only returned to Amazon EC2 when the instance is stopped or terminated
    2. In Amazon VPC, an instance retains its private IP addresses when the instance is stopped.
    3. In Amazon VPC, an instance does NOT retain its private IP addresses when the instance is stopped
    4. In Amazon EC2, the private IP address is associated exclusively with the instance for its lifetime
  7. To ensure failover capabilities, consider using a _____ for incoming traffic on a network interface”.
    1. primary public IP
    2. secondary private IP
    3. secondary public IP
    4. add on secondary IP
  8. Which statements are true about Elastic Network Interface (ENI)? (Choose 2 answers)
    1. You can attach an ENI in one AZ to an instance in another AZ
    2. You can change the security group membership of an ENI
    3. You can attach an instance to tow different subnets within a VPC by using two ENIs
    4. You can attach an ENI in one VPC to an instance in another VPC
  9. A user is planning to host a web server as well as an app server on a single EC2 instance, which is a part of the public subnet of a VPC. How can the user setup to have two separate public IPs and separate security groups for both the application as well as the web server?
    1. Launch a VPC instance with two network interfaces. Assign a separate security group to each and AWS will assign a separate public IP to them. (AWS cannot assign public IPs for instance with multiple ENIs)
    2. Launch VPC with two separate subnets and make the instance a part of both the subnets.
    3. Launch a VPC instance with two network interfaces. Assign a separate security group and elastic IP to them.
    4. Launch a VPC with ELB such that it redirects requests to separate VPC instances of the public subnet.
  10. An organization has created multiple components of a single application for compartmentalization. Currently all the components are hosted on a single EC2 instance. Due to security reasons the organization wants to implement two separate SSLs for the separate modules although it is already using VPC. How can the organization achieve this with a single instance?
    1. Create a VPC instance, which will have both the ACL and the security group attached to it and have separate rules for each IP address.
    2. Create a VPC instance, which will have multiple network interfaces with multiple elastic IP addresses.
    3. You have to launch two instances each in a separate subnet and allow VPC peering for a single IP.
    4. Create a VPC instance, which will have multiple subnets attached to it and each will have a separate IP address.
  11. Your system automatically provisions EIPs to EC2 instances in a VPC on boot. The system provisions the whole VPC and stack at once. You have two of them per VPC. On your new AWS account, your attempt to create a Development environment failed, after successfully creating Staging and Production environments in the same region. What happened?
    1. You didn’t choose the Development version of the AMI you are using.
    2. You didn’t set the Development flag to true when deploying EC2 instances.
    3. You hit the soft limit of 5 EIPs per region and requested a 6th. (There is a soft limit of 5 EIPs per Region for VPC on new accounts. The third environment could not allocate the 6th EIP)
    4. You hit the soft limit of 2 VPCs per region and requested a 3rd.
  12. A user has created a VPC with a public subnet. The user has terminated all the instances, which are part of the subnet. Which of the below mentioned statements is true with respect to this scenario?
    1. The user cannot delete the VPC since the subnet is not deleted
    2. All network interface attached with the instances will be deleted
    3. When the user launches a new instance it cannot use the same subnet
    4. The subnet to which the instances were launched with will be deleted

References

AWS Elastic Cloud Compute – EC2

Elastic Cloud Compute – EC2 Overview

  • Elastic Compute Cloud – EC2 provides scalable computing capacity in AWS
  • EC2 eliminates the need to invest in hardware upfront, so applications can be developed and deployed faster.
  • EC2 can be used to launch as many or as few virtual servers as you need, configure security and networking, and manage storage.
  • EC2 enables you to scale up or down to handle changes in requirements or spikes in popularity, reducing your need to forecast traffic.

EC2 features

  • EC2 instances – Virtual computing environments
  • Amazon Machine Images (AMIs) – Preconfigured templates for the instances that package the bits needed for a server (including the operating system and additional software)
  • Instance types – Various configurations of CPU, memory, storage, and networking capacity for the instances
  • Key Pairs – Secure login information for the instances (AWS stores the public key, and you store the private key in a secure place)
  • Instance Store VolumesStorage volumes for temporary data that are deleted when you stop or terminate your instance, known as
  • EBS Volumes – Persistent storage volumes for the data using Elastic Block Store (Amazon EBS)
  • Regions and Availability ZonesMultiple physical locations for the resources, such as instances and EBS volumes
  • Security GroupsA firewall that enables you to specify the protocols, ports, and source IP ranges that can reach the instances
  • Elastic IP addresses – Static IP addresses for dynamic cloud computing
  • Tags – Metadata can be created and assigned to EC2 resources

Accessing EC2

  • Amazon EC2 console
    • Amazon EC2 console is the web-based user interface that can be accessed from the AWS management console
  • AWS Command line Interface (CLI)
    • Provides commands for a broad set of AWS products, and is supported on Windows, Mac, and Linux.
  • Amazon EC2 Command Line Interface (CLI) tools
    • Provides commands for Amazon EC2, Amazon EBS, and Amazon VPC, and is supported on Windows, Mac, and Linux
  • AWS Tools for Windows Powershell
    • Provides commands for a broad set of AWS products for those who script in the PowerShell environment
  • AWS Query API
    • Query API allows for requests are HTTP or HTTPS requests that use the HTTP verbs GET or POST and a Query parameter named Action
  • AWS SDK libraries
    • AWS provides libraries in various languages which provide basic functions that automate tasks such as cryptographically signing your requests, retrying requests, and handling error responses

Additional Reading

AWS Certification Exam Practice Questions

  • Questions are collected from Internet and the answers are marked as per my knowledge and understanding (which might differ with yours).
  • AWS services are updated everyday and both the answers and questions might be outdated soon, so research accordingly.
  • AWS exam questions are not updated to keep up the pace with AWS updates, so even if the underlying feature has changed the question might not be updated
  • Open to further feedback, discussion and correction.
  1. What are the Amazon EC2 API tools?
    1. They don’t exist. The Amazon EC2 AMI tools, instead, are used to manage permissions.
    2. Command-line tools to the Amazon EC2 web service
    3. They are a set of graphical tools to manage EC2 instances.
    4. They don’t exist. The Amazon API tools are a client interface to Amazon Web Services.
  2. When a user is launching an instance with EC2, which of the below mentioned options is not available during the instance launch console for a key pair?
    1. Proceed without the key pair
    2. Upload a new key pair
    3. Select an existing key pair
    4. Create a new key pair

References

AWS_EC2

AWS EC2 Security – Certification

AWS EC2 Security

EC2 Key Pairs

  • EC2 uses public-key cryptography to encrypt & decrypt login information
  • Public-key cryptography uses a public key to encrypt a piece of data, such as a password, then the recipient uses the private key to decrypt the data.
  • Public and private keys are known as a key pair.
  • To log in to an EC2 instance, a key pair needs to be created and specified when the instance is launched, and the private key can be used to connect to the instance.
  • Linux instances have no password, and the key pair is used for ssh log in
  • For Windows instances, the key pair can be used to obtain the administrator password and then log in using RDP
  • EC2 stores the public key only, and the private key resides with the user. EC2 doesn’t keep a copy of your private key
  • Public key content (on Linux instances) is placed in an entry within  ~/.ssh/authorized_keys at boot time and enables the user to securely access the instance without passwords
  • Public key specified for an instance when launched is also available through its instance metadata http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key
  • EC2 Security Best Practice: Store the private keys in a secure place as anyone who possesses the private key can decrypt the login information
  • Also, if the private key is lost, there is no way to recover the same.
    • For instance store, you cannot access the instance
    • For EBS-backed Linux instances, access can be regained.
      • EBS-backed instance can be stopped, its root volume detached and attached to another instance as a data volume
      • Modify the authorized_keys file, move the volume back to the original instance, and restart the instance
  • Key pair associated with the instances can either be
    • Generated by EC2
      • Keys that EC2 uses are 2048-bit SSH-2 RSA keys.
    • Created separately (using third-party tools) and Imported into EC2
      • EC2 only accepts RSA keys and does not accept DSA keys
      • Supported lengths: 1024, 2048, and 4096
  • supports five thousand key pairs per region
  • Deleting a key pair only deletes the public key and does not impact the servers already launched with the key

EC2 Security Groups

  • An EC2 instance, when launched, can be associated with one or more security groups with the instance, which acts as a virtual firewall that controls the traffic to that instance
  • Security groups helps specify rules that control the inbound traffic that’s allowed to reach the instances and the outbound traffic that’s allowed to leave the instance
  • Security groups are associated with network interfaces. Changing an instance’s security groups changes the security groups associated with the primary network interface (eth0)
  • An Network interface can be associated with 5 security groups and with 50 60 rules per security group
  • Rules for a security group can be modified at any time; the new rules are automatically applied to all instances associated with the security group.
  • All the rules from all associated security groups are evaluated to decide where to allow traffic to an instance
  • Security Group features
    • For the VPC default security group, it allows all inbound traffic from other instances associated with the default security group
    • By default, VPC default security groups or newly created security groups allow all outbound traffic
    • Security group rules are always permissive; deny rules can’t be created
    • Rules can be added and removed any time.
    • Any modification to the rules are automatically applied to the instances associated with the security group after a short period, depending on the connection tracking for the traffic
    • Security groups are stateful — if you send a request from your instance, the response traffic for that request is allowed to flow in regardless of inbound security group rules. For VPC security groups, this also means that responses to allowed inbound traffic are allowed to flow out, regardless of outbound rules
    • If multiple rules are defined for the same protocol and port, the Most permissive rule is applied for e.g. for multiple rules for tcp and port 22 for specific IP and Everyone, everyone is granted access being the most permissive rule

Connection Tracking

  • Security groups are Stateful and they use Connection tracking to track information about traffic to and from the instance.
  • This allows responses to inbound traffic to flow out of the instance regardless of outbound security group rules, and vice versa.
  • Connection Tracking is maintained only if there is no explicit Outbound rule for an Inbound request (and vice versa)
  • However, if there is an explicit Outbound rule for an Inbound request, the response traffic is allowed on the basis of the Outbound rule and not on the Tracking information
  • Any existing flow of traffic, that is tracked, is not interrupted even if the rules for the security groups are changed. To ensure traffic is immediately interrupted, use NACL as they are stateless and therefore do not allow automatic response traffic.
  • Also, If the instance (host A) initiates traffic to host B and uses a protocol other than TCP, UDP, or ICMP,  the instance’s firewall only tracks the IP address and protocol number for the purpose of allowing response traffic from host B. If host B initiates traffic to your instance in a separate request within 600 seconds of the original request or response, your instance accepts it regardless of inbound security group rules, because it’s regarded as response traffic.
  • can be controlled by modifying the security group’s outbound rules to permit only certain types of outbound traffic or using NACL

IAM with EC2

  • IAM policy can be defined to allow or deny a user access to the EC2 resources and actions
  • EC2 partially supports resource-level permissions. For some EC2 API actions, you cannot specify which resource a user is allowed to work with for that action; instead, you have to allow users to work with all resources for that action
  • IAM allows to control only what actions a user can perform on the EC2 resources but cannot be used to grant access for users to be able to access or login to the instances

EC2 with IAM Role

  • EC2 instances can be launched with IAM roles so that the applications can securely make API requests from your instances,
  • IAM roles prevents the need to share as well as manage, rotate the security credentials that the applications use
  • IAM role can be associated with the EC2 instance only when it is launched. Role cannot be assigned to an existing instance
  • IAM role can be added to an existing running EC2 instance, as per the latest enhancement from AWS,
  • EC2 uses an instance profile as a container for an IAM role.
    • Creation of an IAM role using the console, creates an instance profile automatically and gives it the same name as the role it corresponds to.
    • When using the AWS CLI, API, or an AWS SDK to create a role, the role and instance profile needs to be created as separate actions, and they can be given different names.
  • To launch an instance with an IAM role, the name of its instance profile needs to be specified.
  • An application on the instance can retrieve the security credentials provided by the role from the instance metadata item http://169.254.169.254/latest/meta-data/iam/security-credentials/role-name.
  • Security credentials are temporary and are rotated automatically and new credentials are made available at least five minutes prior to the expiration of the old credentials.
  • Best Practice: Always launch EC2 instance with IAM role instead of hardcoded credentials

EC2 IAM Role S3 Access

AWS Certification Exam Practice Questions

  • Questions are collected from Internet and the answers are marked as per my knowledge and understanding (which might differ with yours).
  • AWS services are updated everyday and both the answers and questions might be outdated soon, so research accordingly.
  • AWS exam questions are not updated to keep up the pace with AWS updates, so even if the underlying feature has changed the question might not be updated
  • Open to further feedback, discussion and correction.
  1. You launch an Amazon EC2 instance without an assigned AWS identity and Access Management (IAM) role. Later, you decide that the instance should be running with an IAM role. Which action must you take in order to have a running Amazon EC2 instance with an IAM role assigned to it?
    1. Create an image of the instance, and register the image with an IAM role assigned and an Amazon EBS volume mapping.
    2. Create a new IAM role with the same permissions as an existing IAM role, and assign it to the running instance. (As per AWS latest enhancement, this is possible now)
    3. Create an image of the instance, add a new IAM role with the same permissions as the desired IAM role, and deregister the image with the new role assigned.
    4. Create an image of the instance, and use this image to launch a new instance with the desired IAM role assigned (This was correct before, as it was not possible to add an IAM role to an existing instance)
  2. What does the following command do with respect to the Amazon EC2 security groups? ec2-revoke RevokeSecurityGroupIngress
    1. Removes one or more security groups from a rule.
    2. Removes one or more security groups from an Amazon EC2 instance.
    3. Removes one or more rules from a security group
    4. Removes a security group from our account.
  3. Which of the following cannot be used in Amazon EC2 to control who has access to specific Amazon EC2 instances?
    1. Security Groups
    2. IAM System
    3. SSH keys
    4. Windows passwords
  4. You must assign each server to at least _____ security group
    1. 3
    2. 2
    3. 4
    4. 1
  5. A company is building software on AWS that requires access to various AWS services. Which configuration should be used to ensure that AWS credentials (i.e., Access Key ID/Secret Access Key combination) are not compromised?
    1. Enable Multi-Factor Authentication for your AWS root account.
    2. Assign an IAM role to the Amazon EC2 instance
    3. Store the AWS Access Key ID/Secret Access Key combination in software comments.
    4. Assign an IAM user to the Amazon EC2 Instance.
  6. Which of the following items are required to allow an application deployed on an EC2 instance to write data to a DynamoDB table? Assume that no security keys are allowed to be stored on the EC2 instance. (Choose 2 answers)
    1. Create an IAM Role that allows write access to the DynamoDB table
    2. Add an IAM Role to a running EC2 instance. (As per AWS latest enhancement, this is possible now)
    3. Create an IAM User that allows write access to the DynamoDB table.
    4. Add an IAM User to a running EC2 instance.
    5. Launch an EC2 Instance with the IAM Role included in the launch configuration (This was correct before, as it was not possible to add an IAM role to an existing instance)
  7. You have an application running on an EC2 Instance, which will allow users to download files from a private S3 bucket using a pre-assigned URL. Before generating the URL the application should verify the existence of the file in S3. How should the application use AWS credentials to access the S3 bucket securely?
    1. Use the AWS account access Keys the application retrieves the credentials from the source code of the application.
    2. Create a IAM user for the application with permissions that allow list access to the S3 bucket launch the instance as the IAM user and retrieve the IAM user’s credentials from the EC2 instance user data.
    3. Create an IAM role for EC2 that allows list access to objects in the S3 bucket. Launch the instance with the role, and retrieve the role’s credentials from the EC2 Instance metadata
    4. Create an IAM user for the application with permissions that allow list access to the S3 bucket. The application retrieves the IAM user credentials from a temporary directory with permissions that allow read access only to the application user.
  8. A user has created an application, which will be hosted on EC2. The application makes calls to DynamoDB to fetch certain data. The application is using the DynamoDB SDK to connect with from the EC2 instance. Which of the below mentioned statements is true with respect to the best practice for security in this scenario?
    1. The user should attach an IAM role with DynamoDB access to the EC2 instance
    2. The user should create an IAM user with DynamoDB access and use its credentials within the application to connect with DynamoDB
    3. The user should create an IAM role, which has EC2 access so that it will allow deploying the application
    4. The user should create an IAM user with DynamoDB and EC2 access. Attach the user with the application so that it does not use the root account credentials
  9. Your application is leveraging IAM Roles for EC2 for accessing object stored in S3. Which two of the following IAM policies control access to you S3 objects.
    1. An IAM trust policy allows the EC2 instance to assume an EC2 instance role.
    2. An IAM access policy allows the EC2 role to access S3 objects
    3. An IAM bucket policy allows the EC2 role to access S3 objects. (Bucket policy is defined with S3 and not with IAM)
    4. An IAM trust policy allows applications running on the EC2 instance to assume as EC2 role (Trust policy allows EC2 instance to assume the role)
    5. An IAM trust policy allows applications running on the EC2 instance to access S3 objects. (Applications can access S3 through EC2 assuming the role)
  10. You have an application running on an EC2 Instance, which will allow users to download files from a private S3 bucket using a pre-assigned URL. Before generating the URL the application should verify the existence of the file in S3. How should the application use AWS credentials to access the S3 bucket securely?
    1. Use the AWS account access Keys the application retrieves the credentials from the source code of the application.
    2. Create a IAM user for the application with permissions that allow list access to the S3 bucket launch the instance as the IAM user and retrieve the IAM user’s credentials from the EC2 instance user data.
    3. Create an IAM role for EC2 that allows list access to objects in the S3 bucket. Launch the instance with the role, and retrieve the role’s credentials from the EC2 Instance metadata
    4. Create an IAM user for the application with permissions that allow list access to the S3 bucket. The application retrieves the IAM user credentials from a temporary directory with permissions that allow read access only to the application user.

AWS EC2 Instance Lifecycle

EC2 Instance Lifecycle Overview

  • EC2 instance lifecycle determines how an EC2 instance transitions through different states from the moment it is launched to its termination

EC2 Instance Lifecycle

Instance Launch

  •  Pending
    • When the instance is first launched is enters into the pending state
  • Running
    • After the instance is launched, it enters into the running state
    • Charges are incurred for each second, with a one-minute minimum, that the instance running is running, even if the instance remains idle

Instance Start & Stop (EBS-backed instances only)

  • Only an EBS-backed instance can be stopped and started.
  • Instance store-backed instance cannot be stopped and started.
  • An instance can be stopped & started in case the instance fails a status check or is not running as expected
  • Stop
    • After the instance is stopped, it enters in stopping state and then to stopped state.
    • Charges are only incurred for the EBS storage and not for the instance hourly charge or data transfer.
    • While the instance is stopped, its root volume can be treated like any other volume, and modified for e.g. repair file system problems or update software or change the instance type, user data, EBS optimization attributes, etc
    • Volume can be detached from the stopped instance, and attached to a running instance, modified, detached from the running instance, and then reattached to the stopped instance. It should be reattached using the storage device name that’s specified as the root device in the block device mapping for the instance.
  • Start
    • When the instance is started, it enters into pending state and then into running
    • An instance, when stopped and started, is launched on a new host
    • Any data on an instance store volume (not root volume) would be lost while data on the EBS volume persists
  • EC2 instance retains its private IP address as well as the Elastic IP address.
  • If the instance has an IPv6 address, it retains its IPv6 address.
  • However, the public IP address, if assigned instead of the Elastic IP address, would be released
  • For each transition of an instance from stopped to running, charges per second are incurred when the instance is running, with a minimum of one minute every time the instance is started

Instance Hibernate

  • Instance hibernation signals the operating system to perform hibernation (suspend-to-disk), which saves the contents from the instance memory (RAM) to the EBS root volume
  • Instance’s EBS root volume and any attached EBS data volumes are persisted, including the saved contents of the RAM.
  • Any EC2 instance store volumes remain attached to the instance, but the data on the instance store volumes is lost.
  • When the instance is restarted, the EBS root volume is restored to its previous state and the RAM contents are reloaded. Previously attached data volumes are reattached and the instance retains its instance ID.
  • After the instance is hibernated, it enters in stopping state and then to stopped state.
  • When the instance is restarted
    • It enters the pending state and the instance is moved to a new host computer (though in some cases, it remains on the current host).
    • EBS root volume is restored to its previous state
    • RAM contents are reloaded
    • Processes that were previously running on the instance are resumed
    • Previously attached data volumes are reattached and the instance retains its instance ID
    • Instance retains private IPv4 addresses and any IPv6 addresses
    • Instance retains its Elastic IP address
    • Instance releases its Public IPv4 address and would get a new one
  • Hibernation prerequisites
    • Supported instance families – C3, C4, C5, M3, M4, M5, R3, R4, R5, & T2
    • Instance RAM size – must be less than 150 GB.
    • Instance size – not supported for bare metal instances.
    • Supported AMIs must be an HVM AMI that supports hibernation
    • Root volume type – must be EBS volume and not instance store
    • EBS root volume size – must be large enough to store the RAM contents
    • EBS root volume MUST be encrypted to ensure the protection of sensitive content that is in memory at the time of hibernation
    • Enable hibernation at launch, as changing it is not supported on an existing instance
    • Purchasing options – Only On-Demand Instances and Reserved Instances supported
  • Limitations or Unsupported Actions
    • Changing the instance type or size of a hibernated instance
    • Creating snapshots or AMIs from hibernated instances or instances for which hibernation is enabled
    • the data on any instance store volumes is lost
    • can’t hibernate an instance that has more than 150 GB of RAM.
    • can’t hibernate an instance that is in an Auto Scaling group or used by ECS. If the instance is in an Auto Scaling group is hibernated, the EC2 Auto Scaling service marks the stopped instance as unhealthy, and may terminate it and launch a replacement instance.
    • An instance cannot be hibernated for more than 60 days.

Instance Reboot

  • Both EBS-backed and Instance store-backed instances can be rebooted
  • An instance remains on the same host computer and maintains its public DNS name, private IP address
  • Data on the EBS and Instance store volume is also retained
  • AWS recommends using EC2 to reboot the instance instead of running the operating system reboot command from the instance as it performs a hard reboot if the instance does not cleanly shut down within four minutes also creates an API record in CloudTrail if enabled.

Instance Retirement

  • An instance is scheduled to be retired when AWS detects an irreparable failure of the underlying hardware hosting the instance.
  • When an instance reaches its scheduled retirement date, it is stopped or terminated by AWS.
  • If the instance root device is an EBS volume, the instance is stopped and can be started again at any time.
  • If the instance root device is an instance store volume, the instance is terminated, and cannot be used again.

Instance Termination

  • An instance can be terminated, and it enters into the shutting-down, and then the terminated state
  • After an instance is terminated, it can’t be connected and no charges are incurred
  • Instance Shutdown behavior
    • Each EBS-backed instance supports the InstanceInitiatedShutdownBehavior attribute which determines whether the instance would be stopped or terminated when a shutdown command is initiated from the instance itself for e.g. shutdown, halt or poweroff command in linux
    • Default behavior for the instance to be stopped.
    • A shutdown command for an Instance store-backed instance will always terminate the instance
  • Termination protection
    • Termination protection ( DisableApiTermination attribute) can be enabled on the instance to prevent it from being accidentally terminated
    • DisableApiTermination from the Console, CLI or API.
    • Instance can be terminated through EC2 CLI.
    • Termination protection does not work for instances when
      • part of an Autoscaling group
      • launched as Spot instances
      • terminating an instance by initiating shutdown from the instance
  • Data persistence
    • EBS volume has a DeleteOnTermination  attribute which determines whether the volumes would be persisted or deleted when an instance they are associated with are terminated
    • Data on Instance store volume data does not persist
    • Default is to delete the root device volume and preserve any other EBS volumes. i.e.
      • Data on EBS root volumes have the DeleteOnTermination flag set to true and would be deleted by default
      • Additional EBS volumes attached have the DeleteOnTermination flag set to false are not deleted but just detached from the instance

EC2 Instance Lifecycle States and Billing

AWS Certification Exam Practice Questions

  • Questions are collected from Internet and the answers are marked as per my knowledge and understanding (which might differ with yours).
  • AWS services are updated everyday and both the answers and questions might be outdated soon, so research accordingly.
  • AWS exam questions are not updated to keep up the pace with AWS updates, so even if the underlying feature has changed the question might not be updated
  • Open to further feedback, discussion and correction.
  1. What does Amazon EC2 provide?
    1. Virtual servers in the Cloud
    2. A platform to run code (Java, PHP, Python), paying on an hourly basis.
    3. Computer Clusters in the Cloud.
    4. Physical servers, remotely managed by the customer.
  2. A user has enabled termination protection on an EC2 instance. The user has also set Instance initiated shutdown behavior to terminate. When the user shuts down the instance from the OS, what will happen?
    1. The OS will shutdown but the instance will not be terminated due to protection
    2. It will terminate the instance
    3. It will not allow the user to shutdown the instance from the OS
    4. It is not possible to set the termination protection when an Instance initiated shutdown is set to Terminate
  3. A user has launched an EC2 instance and deployed a production application in it. The user wants to prohibit any mistakes from the production team to avoid accidental termination. How can the user achieve this?
    1. The user can the set DisableApiTermination attribute to avoid accidental termination
    2. It is not possible to avoid accidental termination
    3. The user can set the Deletion termination flag to avoid accidental termination
    4. The user can set the InstanceInitiatedShutdownBehavior flag to avoid accidental termination
  4. You have been doing a lot of testing of your VPC Network by deliberately failing EC2 instances to test whether instances are failing over properly. Your customer who will be paying the AWS bill for all this asks you if he being charged for all these instances. You try to explain to him how the billing works on EC2 instances to the best of your knowledge. What would be an appropriate response to give to the customer in regards to this?
    1. Billing commences when Amazon EC2 AMI instance is completely up and billing ends as soon as the instance starts to shutdown.
    2. Billing commences when Amazon EC2 initiates the boot sequence of an AMI instance and billing ends when the instance shuts down.
    3. Billing only commences only after 1 hour of uptime and billing ends when the instance terminates.
    4. Billing commences when Amazon EC2 initiates the boot sequence of an AMI instance and billing ends as soon as the instance starts to shutdown.

References