Lambda

AWS Lambda Functions

~ Last updated on : ~ jayendrapatil ~ 1 Comment

AWS Lambda Functions

  • Each function has associated configuration information, such as its name, description, runtime, entry point, and resource requirements
  • Lambda functions should be designed as stateless
    • to allow launching of as many copies of the function as needed as per the demand.
    • Local file system access, child processes, and similar artifacts may not extend beyond the lifetime of the request
    • The state can be maintained externally in DynamoDB or S3
  • Lambda Execution role can be assigned to the function to grant permission to access other resources.
  • Functions have the following restrictions
    • Inbound network connections are blocked
    • Outbound connections only TCP/IP sockets are supported
    • ptrace (debugging) system calls are blocked
    • TCP port 25 traffic is also blocked as an anti-spam measure.
  • Lambda may choose to retain an instance of the function and reuse it to serve a subsequent request, rather than creating a new copy.
  • Lambda Layers provide a convenient way to package libraries and other dependencies that you can use with your Lambda functions.
  • Function versions can be used to manage the deployment of the functions.
  • Function Alias supports creating aliases, which are mutable, for each function version.
  • Functions have the following limits
    • RAM – 128 MB to 10,240 MB (10 GB)
    • CPU is linked to RAM and cannot be set manually.
      • 2 vCPUs = 1769 MB RAM
      • 6 vCPUs = 10240 MB RAM
    • Timeout – 900 Secs or 15 mins
    • /tmp storage between 512 MB and 10,240 MB
    • Deployment Package – 50 MB (zipped), 250 MB (unzipped) including layers
    • Concurrent Executions – 1000 (soft limit)
    • Container Image Size – 10 GB
    • Invocation Payload (request/response) – 6 MB (sync), 256 KB (async)
  • Functions are automatically monitored, and real-time metrics are reported through CloudWatch, including total requests, latency, error rates, and throttled requests.
  • Lambda automatically integrates with CloudWatch logs, creating a log group for each function and providing basic application lifecycle event log entries, including logging the resources consumed for each use of that function.
  • Functions support code written in
    • Node.js (JavaScript)
    • Python
    • Ruby
    • Java (Java 8 compatible)
    • C# (.NET Core)
    • Go
    • Custom runtime
  • Container images are also supported.
  • Failure Handling
    • For S3 bucket notifications and custom events, Lambda will attempt execution of the function three times in the event of an error condition in the code or if a service or resource limit is exceeded.
    • For ordered event sources that Lambda polls, e.g. DynamoDB Streams and Kinesis streams, it will continue attempting execution in the event of a developer code error until the data expires.
    • Kinesis and DynamoDB Streams retain data for a minimum of 24 hours
    • Dead Letter Queues (SNS or SQS) can be configured for events to be placed, once the retry policy for asynchronous invocations is exceeded

Lambda Layers

  • Lambda Layers provide a convenient way to package libraries and other dependencies that you can use with your Lambda functions.
  • Layers help reduce the size of uploaded deployment archives and make it faster to deploy your code.
  • A layer is a .zip file archive that can contain additional code or data.
  • A layer can contain libraries, a custom runtime, data, or configuration files.
  • Layers promote reusability, code sharing, and separation of responsibilities so that you can iterate faster on writing business logic.
  • Layers can be used only with Lambda functions deployed as a .zip file archive.
  • For functions defined as a container image, the preferred runtime and all code dependencies can be packaged when the container image is created.
  • A Layer can be created by bundling the content into a .zip file archive and uploading the .zip file archive to the layer from S3 or the local machine.
  • Lambda extracts the layer contents into the /opt directory when setting up the execution environment for the function.

Environment Variables

  • Environment variables can be used to adjust the function’s behavior without updating the code.
  • An environment variable is a pair of strings that are stored in a function’s version-specific configuration.
  • The Lambda runtime makes environment variables available to the code and sets additional environment variables that contain information about the function and invocation request.
  • Environment variables are not evaluated prior to the function invocation.
  • Lambda stores environment variables securely by encrypting them at rest.
  • AWS recommends using Secrets Manager instead of storing secrets in the environment variables.

Lambda Function Limits

  • RAM – 128 MB to 10,240 MB (10 GB)
  • CPU is linked to RAM and cannot be set manually.
    • 2 vCPUs = 1769 MB RAM
    • 6 vCPUs = 10240 MB RAM
  • Timeout – 900 Secs or 15 mins
  • /tmp storage between 512 MB and 10,240 MB
  • Deployment Package – 50 MB (zipped), 250 MB (unzipped) including layers
  • Concurrent Executions – 1000 (soft limit)
  • Container Image Size – 10 GB
  • Invocation Payload (request/response) – 6 MB (sync), 256 KB (async)

Lambda Functions Versioning

  • Function versions can be used to manage the deployment of the functions.
  • Each function has a single, current version of the code.
  • Lambda creates a new version of the function each time it’s published.
  • A function version includes the following information:
    • The function code and all associated dependencies.
    • The Lambda runtime that invokes the function.
    • All the function settings, including the environment variables.
    • A unique Amazon Resource Name (ARN) to identify the specific version of the function.
  • Function versions are immutable, however, support Aliases which are mutable.

Lambda Functions Alias

  • Lambda supports creating aliases, which are mutable, for each function version.
  • Alias is a pointer to a specific function version, with a unique ARN.
  • Each alias maintains an ARN for a function version to which it points.
  • An alias can only point to a function version, not to another alias
  • Alias helps in rolling out new changes or rolling back to old versions
  • Alias supports routing configuration to point to a maximum of two Lambda function versions. It can be used for canary testing to send a portion of traffic to a second function version.

References

AWS_Lambda_Functions

AWS Lambda

~ Last updated on : ~ jayendrapatil ~ 19 Comments

AWS Lambda

  • AWS Lambda offers Serverless computing that allows applications and services to be built and run without thinking about servers.
  • With serverless computing, the application still runs on servers, but all the server management is done by AWS.
  • helps run code without provisioning or managing servers, where you pay only for the compute time when the code is running.
  • is priced on a pay-per-use basis and there are no charges when the code is not running.
  • allows the running of code for any type of application or backend service with zero administration.
  • performs all the operational and administrative activities on your behalf, including capacity provisioning, monitoring fleet health, applying security patches to the underlying compute resources, deploying code, running a web service front end, and monitoring and logging the code.
  • does not provide access to the underlying compute infrastructure.
  • handles scalability and availability as it
    • provides easy scaling and high availability to the code without additional effort on your part.
    • is designed to process events within milliseconds.
    • is designed to run many instances of the functions in parallel.
    • is designed to use replication and redundancy to provide high availability for both the service and the functions it operates.
    • has no maintenance windows or scheduled downtimes for either.
    • has a default safety throttle for the number of concurrent executions per account per region.
    • has a higher latency immediately after a function is created, or updated, or if it has not been used recently.
    • for any function updates, there is a brief window of time, less than a minute, when requests would be served by both versions
  • Security
    • stores code in S3 and encrypts it at rest and performs additional integrity checks while the code is in use.
    • each function runs in its own isolated environment, with its own resources and file system view
    • supports Code Signing using AWS Signer, which offers trust and integrity controls that enable you to verify that only unaltered code from approved developers is deployed in the functions. 
  • Functions must complete execution within 900 seconds. The default timeout is 3 seconds. The timeout can be set the timeout to any value between 1 and 900 seconds.
  • AWS Step Functions can help coordinate a series of Lambda functions in a specific order. Multiple functions can be invoked sequentially, passing the output of one to the other, and/or in parallel, while the state is being maintained by Step Functions.
  • AWS X-Ray helps to trace functions, which provides insights such as service overhead, function init time, and function execution time.
  • Lambda Provisioned Concurrency provides greater control over the performance of serverless applications.
  • Lambda@Edge allows you to run code across AWS locations globally without provisioning or managing servers, responding to end-users at the lowest network latency.
  • Lambda Extensions allow integration of Lambda with other third-party tools for monitoring, observability, security, and governance.
  • Compute Savings Plan can help save money for Lambda executions.
  • CodePipeline and CodeDeploy can be used to automate the serverless application release process.
  • RDS Proxy provides a highly available database proxy that manages thousands of concurrent connections to relational databases.
  • Supports Elastic File Store , to provide a shared, external, persistent, scalable volume using a fully managed elastic NFS file system without the need for provisioning or capacity management.
  • supports Function URLs, a built-in HTTPS endpoint that can be invoked using the browser, curl, and any HTTP client. 

Functions & Event Sources

  • Core components of Lambda are functions and event sources.
    • Event source – an AWS service or custom application that publishes events.
    • Function – a custom code that processes the events.

Lambda Functions

  • Each function has associated configuration information, such as its name, description, runtime, entry point, and resource requirements
  • Lambda functions should be designed as stateless
    • to allow launching of as many copies of the function as needed as per the demand.
    • Local file system access, child processes, and similar artifacts may not extend beyond the lifetime of the request
    • The state can be maintained externally in DynamoDB or S3
  • Lambda Execution role can be assigned to the function to grant permission to access other resources.
  • Functions have the following restrictions
    • Inbound network connections are blocked
    • Outbound connections only TCP/IP sockets are supported
    • ptrace (debugging) system calls are blocked
    • TCP port 25 traffic is also blocked as an anti-spam measure.
  • Lambda may choose to retain an instance of the function and reuse it to serve a subsequent request, rather than creating a new copy.
  • Lambda Layers provide a convenient way to package libraries and other dependencies that you can use with your Lambda functions.
  • Function versions can be used to manage the deployment of the functions.
  • Function Alias supports creating aliases, which are mutable, for each function version.
  • Functions are automatically monitored, and real-time metrics are reported through CloudWatch, including total requests, latency, error rates, etc.
  • Lambda automatically integrates with CloudWatch logs, creating a log group for each function and providing basic application lifecycle event log entries, including logging the resources consumed for each use of that function.
  • Functions support code written in
    • Node.js (JavaScript)
    • Python
    • Ruby
    • Java (Java 8 compatible)
    • C# (.NET Core)
    • Go
    • Custom runtime
  • Container images are also supported.
  • Failure Handling
    • For S3 bucket notifications and custom events, Lambda will attempt execution of the function three times in the event of an error condition in the code or if a service or resource limit is exceeded.
    • For ordered event sources that Lambda polls, e.g. DynamoDB Streams and Kinesis streams, it will continue attempting execution in the event of a developer code error until the data expires.
    • Kinesis and DynamoDB Streams retain data for a minimum of 24 hours
    • Dead Letter Queues (SNS or SQS) can be configured for events to be placed, once the retry policy for asynchronous invocations is exceeded

Read in-depth  @ Lambda Functions

Lambda Event Sources

  • Event Source is an AWS service or developer-created application that produces events that trigger an AWS Lambda function to run
  • Event source mapping refers to the configuration which maps an event source to a Lambda function.
  • Event sources can be both push and pull sources
    • Services like S3, and SNS publish events to Lambda by invoking the cloud function directly.
    • Lambda can also poll resources in services like Kafka, and Kinesis streams that do not publish events to Lambda.

Read in-depth @ Event Sources

Lambda Execution Environment

  • Lambda invokes the function in an execution environment, which provides a secure and isolated runtime environment.
  • Execution Context is a temporary runtime environment that initializes any external dependencies of the Lambda function code, e.g. database connections or HTTP endpoints.
  • When a function is invoked, the Execution environment is launched based on the provided configuration settings i.e. memory and execution time.
  • After a Lambda function is executed, Lambda maintains the execution environment for some time in anticipation of another function invocation which allows it to reuse the /tmp directory and objects declared outside of the function’s handler method e.g. database connection.
  • When a Lambda function is invoked for the first time or after it has been updated there is latency for bootstrapping as Lambda tries to reuse the Execution Context for subsequent invocations of the Lambda function
  • Subsequent invocations perform better performance as there is no need to “cold-start” or initialize those external dependencies
  • Execution environment
    • takes care of provisioning and managing the resources needed to run the function.
    • provides lifecycle support for the function’s runtime and any external extensions associated with the function.
  • Function’s runtime communicates with Lambda using the Runtime API.
  • Extensions communicate with Lambda using the Extensions API.
  • Extensions can also receive log messages from the function by subscribing to logs using the Logs API.
  • Lambda manages Execution Environment creations and deletion, there is no AWS Lambda API to manage Execution Environment.

Lambda Execution Environment

Lambda in VPC

  • Lambda function always runs inside a VPC owned by the Lambda service which isn’t connected to your account’s default VPC
  • Lambda applies network access and security rules to this VPC and maintains and monitors the VPC automatically.
  • A function can be configured to be launched in private subnets in a VPC in your AWS account.
  • Function connected to VPC can access private resources databases, cache instances, or internal services during the execution.
  • To enable the function to access resources inside the private VPC, additional VPC-specific configuration information that includes private subnet IDs and security group IDs must be provided.
  • Lambda uses this information to set up ENIs that enables the function to connect securely to other resources within your private VPC.
  • Functions connected to VPC can’t access the Internet and need a NAT Gateway to access any external resources outside of AWS.
  • Functions cannot connect directly to a VPC with dedicated instance tenancy, instead, peer it to a second VPC with default tenancy.

Lambda Security

  • All data stored in ephemeral storage is encrypted at rest with a key managed by AWS.
  • Lambda functions provide access only to a single VPC. If multiple subnets are specified, they must all be in the same VPC. Other VPCs can be connected using VPC Peering.
  • Supports Code Signing using AWS Signer, which offers trust and integrity controls that enable you to verify that only unaltered code from approved developers is deployed in the functions. 
  • AWS Lambda can perform the following signature checks at deployment:
    • Corrupt signature – This occurs if the code artifact has been altered since signing.
    • Mismatched signature – This occurs if the code artifact is signed by a signing profile that is not approved.
    • Expired signature – This occurs if the signature is past the configured expiry date.
    • Revoked signature – This occurs if the signing profile owner revokes the signing jobs.
  • For sensitive information, for e.g. passwords, AWS recommends using client-side encryption using AWS Key Management Service – KMS and store the resulting values as ciphertext in your environment variable.
  • Function code should include the logic to decrypt these values.

Lambda Permissions

  • IAM – Use IAM to manage access to the Lambda API and resources like functions and layers.
  • Execution Role – A Lambda function can be provided with an Execution Role, that grants it permission to access AWS services and resources e.g. send logs to CloudWatch and upload trace data to AWS X-Ray.
  • Function Policy – Resource-based Policies
    • Use resource-based policies to give other accounts and AWS services permission to use the Lambda resources.
    • Resource-based permissions policies are supported for functions and layers.

Invoking Lambda Functions

  • Lambda functions can be invoked
    • directly using the Lambda console or API, a function URL HTTP(S) endpoint, an AWS SDK, the AWS CLI, and AWS toolkits.
    • other AWS services like S3 and SNS invoke the function.
    • to read from a stream or queue and invoke the function.
  • Functions can be invoked
    • Synchronously
      • You wait for the function to process the event and return a response.
      • Error handling and retries need to be handled by the Client.
      • Invocation includes API, and SDK for calls from API Gateway.
    • Asynchronously
      • queues the event for processing and returns a response immediately.
      • handles retries and can send invocation records to a destination for successful and failed events.
      • Invocation includes S3, SNS, and CloudWatch Events
      • can define DLQ for handling failed events. AWS recommends using destination instead of DLQ.

Lambda Provisioned Concurrency

  • Lambda Provisioned Concurrency provides greater control over the performance of serverless applications.
  • When enabled, Provisioned Concurrency keeps functions initialized and hyper-ready to respond in double-digit milliseconds.
  • Provisioned Concurrency is ideal for building latency-sensitive applications, such as web or mobile backends, synchronously invoked APIs, and interactive microservices.
  • The amount of concurrency can be increased during times of high demand and lowered or turn it off completely when demand decreases.
  • If the concurrency of a function reaches the configured level, subsequent invocations of the function have the latency and scale characteristics of regular functions.

Lambda@Edge

Read in-depth @ Lambda@Edge

Lambda Extensions

  • Lambda Extensions allow integration of Lambda with other third-party tools for monitoring, observability, security, and governance.

Lambda Best Practices

  • Lambda function code should be stateless and ensure there is no affinity between the code and the underlying compute infrastructure.
  • Instantiate AWS clients outside the scope of the handler to take advantage of connection re-use.
  • Make sure you have set +rx permissions on your files in the uploaded ZIP to ensure Lambda can execute code on your behalf.
  • Lower costs and improve performance by minimizing the use of startup code not directly related to processing the current event.
  • Use the built-in CloudWatch monitoring of the Lambda functions to view and optimize request latencies.
  • Delete old Lambda functions that you are no longer using.

AWS Certification Exam Practice Questions

  • Questions are collected from Internet and the answers are marked as per my knowledge and understanding (which might differ with yours).
  • AWS services are updated everyday and both the answers and questions might be outdated soon, so research accordingly.
  • AWS exam questions are not updated to keep up the pace with AWS updates, so even if the underlying feature has changed the question might not be updated
  • Open to further feedback, discussion and correction.
  • Questions are collected from Internet and the answers are marked as per my knowledge and understanding (which might differ with yours).
  • AWS services are updated everyday and both the answers and questions might be outdated soon, so research accordingly.
  • AWS exam questions are not updated to keep up the pace with AWS updates, so even if the underlying feature has changed the question might not be updated
  • Open to further feedback, discussion and correction.
  1. Your serverless architecture using AWS API Gateway, AWS Lambda, and AWS DynamoDB experienced a large increase in traffic to a sustained 400 requests per second, and dramatically increased in failure rates. Your requests, during normal operation, last 500 milliseconds on average. Your DynamoDB table did not exceed 50% of provisioned throughput, and Table primary keys are designed correctly. What is the most likely issue?
    1. Your API Gateway deployment is throttling your requests.
    2. Your AWS API Gateway Deployment is bottlenecking on request (de)serialization.
    3. You did not request a limit increase on concurrent Lambda function executions. (Refer link – AWS API Gateway by default throttles at 500 requests per second steady-state, and 1000 requests per second at spike. Lambda, by default, throttles at 100 concurrent requests for safety. At 500 milliseconds (half of a second) per request, you can expect to support 200 requests per second at 100 concurrency. This is less than the 400 requests per second your system now requires. Make a limit increase request via the AWS Support Console.)
    4. You used Consistent Read requests on DynamoDB and are experiencing semaphore lock.

AWS Lambda Event Source

~ Last updated on : ~ jayendrapatil ~ 5 Comments

AWS Lambda Event Source

  • Lambda Event Source is an AWS service or developer-created application that produces events that trigger an AWS Lambda function to run.
  • Event sources can be either AWS Services or Custom applications.
  • Event sources can be both push and pull sources
    • Services like S3, and SNS publish events to Lambda by invoking the cloud function directly.
    • Lambda can also poll resources in services like Kafka, and Kinesis streams that do not publish events to Lambda.
  • Events are passed to a Lambda function as an event input parameter. For batch event sources, such as Kinesis Streams, the event parameter may contain multiple events in a single call, based on the requested batch size

Lambda Event Source Mapping

  • Lambda Event source mapping refers to the configuration which maps an event source to a Lambda function.
  • Event source mapping
    • enables automatic invocation of the Lambda function when events occur.
    • identifies the type of events to publish and the Lambda function to invoke when events occur.

Lambda Event Sources Type

AWS Lambda Event Source Types

Push-based

  • also referred to as the Push model
  • includes services like S3, SNS, SES, etc.
  • Event source mapping maintained on the event source side
  • as the event sources invoke the Lambda function, a resource-based policy should be used to grant the event source the necessary permissions.

Pull-based

  • also referred to as the Pull model
  • covers mostly the Stream-based event sources like DynamoDB, Kinesis streams, MQ, SQS, Kafka
  • Event source mapping maintained on the Lambda side

Lambda Event Sources Invocation Model

Synchronously

  • You wait for the function to process the event and return a response.
  • Error handling and retries need to be handled by the Client.
  • Invocation includes API, and SDK for calls from API Gateway.

Asynchronously

  • queues the event for processing and returns a response immediately.
  • handles retries and can send invocation records to a destination for successful and failed events.
  • Invocation includes S3, SNS, and CloudWatch Events

Lambda Supported Event Sources

AWS Lambda can be configured as an event source for multiple AWS services

Service Method of invocation
Amazon Alexa Event-driven; synchronous invocation
Amazon MSK – Managed Streaming for Apache Kafka Lambda polling
Self-managed Apache Kafka Lambda polling
Amazon API Gateway Event-driven; synchronous invocation
AWS CloudFormation Event-driven; asynchronous invocation
Amazon CloudFront (Lambda@Edge) Event-driven; synchronous invocation
Amazon EventBridge (CloudWatch Events) Event-driven; asynchronous invocation
Amazon CloudWatch Logs Event-driven; asynchronous invocation
AWS CodeCommit Event-driven; asynchronous invocation
AWS CodePipeline Event-driven; asynchronous invocation
Amazon Cognito Event-driven; synchronous invocation
AWS Config Event-driven; asynchronous invocation
Amazon Connect Event-driven; synchronous invocation
Amazon DynamoDB Lambda polling
Amazon Elastic File System Special integration
Elastic Load Balancing (Application Load Balancer) Event-driven; synchronous invocation
AWS IoT Event-driven; asynchronous invocation
AWS IoT Events Event-driven; asynchronous invocation
Amazon Kinesis Lambda polling
Amazon Kinesis Data Firehose Event-driven; synchronous invocation
Amazon Lex Event-driven; synchronous invocation
Amazon MQ Lambda polling
Amazon Simple Email Service Event-driven; asynchronous invocation
Amazon Simple Notification Service Event-driven; asynchronous invocation
Amazon Simple Queue Service Lambda polling
Amazon S3 Event-driven; asynchronous invocation
Amazon Simple Storage Service Batch Event-driven; synchronous invocation
Secrets Manager Event-driven; synchronous invocation
AWS X-Ray Special integration

Amazon S3

  • S3 bucket events, such as the object-created or object-deleted events can be processed using Lambda functions for e.g., the Lambda function can be invoked when a user uploads a photo to a bucket to read the image and create a thumbnail.
  • S3 bucket notification configuration feature can be configured for the event source mapping, to identify the S3 bucket events and the Lambda function to invoke.
  • Error handling for an event source depends on how Lambda is invoked
  • S3 invokes your Lambda function asynchronously.

DynamoDB

  • Lambda functions can be used as triggers for the DynamoDB table to take custom actions in response to updates made to the DynamoDB table.
  • Trigger can be created by
    • Enabling DynamoDB Streams for the table.
    • Lambda polls the stream and processes any updates published to the stream
  • DynamoDB is a stream-based event source and with stream-based service, the event source mapping is created in Lambda, identifying the stream to poll and which Lambda function to invoke.
  • Error handling for an event source depends on how Lambda is invoked

Kinesis Streams

  • AWS Lambda can be configured to automatically poll the Kinesis stream periodically (once per second) for new records.
  • Lambda can process any new records such as social media feeds, IT logs, website click streams, financial transactions, and location-tracking events
  • Kinesis Streams is a stream-based event source and with stream-based service, the event source mapping is created in Lambda, identifying the stream to poll and which Lambda function to invoke.
  • Error handling for an event source depends on how Lambda is invoked

Simple Notification Service – SNS

  • SNS notifications can be processed using Lambda
  • When a message is published to an SNS topic, the service can invoke Lambda function by passing the message payload as parameter, which can then process the event
  • Lambda function can be triggered in response to CloudWatch alarms and other AWS services that use SNS.
  • SNS via topic subscription configuration feature can be used for the event source mapping, to identify the SNS topic and the Lambda function to invoke
  • Error handling for an event source depends on how Lambda is invoked
  • SNS invokes your Lambda function asynchronously.

Simple Email Service – SES

  • SES can be used to receive messages and can be configured to invoke Lambda function when messages arrive, by passing in the incoming email event as parameter
  • SES using the rule configuration feature can be used for the event source mapping
  • Error handling for an event source depends on how Lambda is invoked
  • SES invokes your Lambda function asynchronously.

Amazon Cognito

  • Cognito Events feature enables Lambda function to run in response to events in Cognito for e.g. Lambda function can be invoked for the Sync Trigger events, that is published each time a dataset is synchronized.
  • Cognito event subscription configuration feature can be used for the event source mapping
  • Error handling for an event source depends on how Lambda is invoked
  • Cognito is configured to invoke a Lambda function synchronously

CloudFormation

  • Lambda function can be specified as a custom resource to execute any custom commands as a part of deploying CloudFormation stacks and can be invoked whenever the stacks are created, updated, or deleted.
  • CloudFormation using stack definition can be used for the event source mapping
  • Error handling for an event source depends on how Lambda is invoked
  • CloudFormation invokes the Lambda function asynchronously

CloudWatch Logs

  • Lambda functions can be used to perform custom analysis on CloudWatch Logs using CloudWatch Logs subscriptions.
  • CloudWatch Logs subscriptions provide access to a real-time feed of log events from CloudWatch Logs and deliver it to the AWS Lambda function for custom processing, analysis, or loading to other systems.
  • CloudWatch Logs using the log subscription configuration can be used for the event source mapping
  • Error handling for an event source depends on how Lambda is invoked
  • CloudWatch Logs invokes the Lambda function asynchronously

CloudWatch Events

  • CloudWatch Events help respond to state changes in the AWS resources. When the resources change state, they automatically send events into an event stream.
  • Rules that match selected events in the stream can be created to route them to the Lambda function to take action for e.g., the Lambda function can be invoked to log the state of an EC2 instance or AutoScaling Group.
  • CloudWatch Events by using a rule target definition can be used for the event source mapping
  • Error handling for an event source depends on how Lambda is invoked
  • CloudWatch Events invokes the Lambda function asynchronously

CodeCommit

  • Trigger can be created for a CodeCommit repository so that events in the repository will invoke a Lambda function for e.g., Lambda function can be invoked when a branch or tag is created or when a push is made to an existing branch.
  • CodeCommit by using a repository trigger can be used for the event source mapping
  • Error handling for an event source depends on how Lambda is invoked
  • CodeCommit Events invokes the Lambda function asynchronously

Scheduled Events (powered by CloudWatch Events)

  • AWS Lambda can be invoked regularly on a scheduled basis using the schedule event capability in CloudWatch Events.
  • CloudWatch Events by using a rule target definition can be used for the event source mapping
  • Error handling for an event source depends on how Lambda is invoked
  • CloudWatch Events invokes the Lambda function asynchronously

AWS Config

  • Lambda functions can be used to evaluate whether the AWS resource configurations comply with custom Config rules.
  • As resources are created, deleted, or changed, AWS Config records these changes and sends the information to the Lambda functions, which can then evaluate the changes and report results to AWS Config. AWS Config can be used to assess overall resource compliance
  • AWS Config by using a rule target definition can be used for the event source mapping
  • Error handling for an event source depends on how Lambda is invoked
  • AWS Config invokes the Lambda function asynchronously

Amazon API Gateway

  • Lambda function can be invoked over HTTPS by defining a custom REST API and endpoint using Amazon API Gateway.
  • Individual API operations, such as GET and PUT, can be mapped to specific Lambda functions.
  • When an HTTPS request to the API endpoint is received, the API Gateway service invokes the corresponding Lambda function.
  • Error handling for an event source depends on how Lambda is invoked.
  • API Gateway is configured to invoke a Lambda function synchronously.

Other Event Sources: Invoking a Lambda Function On Demand

  • Lambda functions can be invoked on-demand without the need to preconfigure any event source mapping in this case.

AWS Certification Exam Practice Questions

  • Questions are collected from Internet and the answers are marked as per my knowledge and understanding (which might differ with yours).
  • AWS services are updated everyday and both the answers and questions might be outdated soon, so research accordingly.
  • AWS exam questions are not updated to keep up the pace with AWS updates, so even if the underlying feature has changed the question might not be updated
  • Open to further feedback, discussion and correction.

References

AWS_Lambda_Developer_Guide

AWS Compute Services Cheat Sheet

~ Last updated on : ~ jayendrapatil ~ 22 Comments

AWS Compute Services Cheat Sheet

AWS Compute Services

Elastic Cloud Compute – EC2

  • provides scalable computing capacity
  • Features
    • Virtual computing environments, known as EC2 instances
    • Preconfigured templates for EC2 instances, known as Amazon Machine Images (AMIs), that package the bits needed for the server (including the operating system and additional software)
    • Various configurations of CPU, memory, storage, and networking capacity for your instances, known as Instance types
    • Secure login information for your instances using key pairs (public-private keys where private is kept by user)
    • Storage volumes for temporary data that’s deleted when you stop or terminate your instance, known as Instance store volumes
    • Persistent storage volumes for data using Elastic Block Store (EBS)
    • Multiple physical locations for your resources, such as instances and EBS volumes, known as Regions and Availability Zones
    • A firewall to specify the protocols, ports, and source IP ranges that can reach your instances using Security Groups
    • Static IP addresses, known as Elastic IP addresses
    • Metadata, known as tags, can be created and assigned to EC2 resources
    • Virtual networks that are logically isolated from the rest of the AWS cloud, and can optionally connect to on-premises network, known as Virtual private clouds (VPCs)

Amazon Machine Image – AMI

    • template from which EC2 instances can be launched quickly
    • does NOT span across regions, and needs to be copied
    • can be shared with other specific AWS accounts or made public

Instance Types

  • T for applications needing general usage
    • T2 instances are Burstable Performance Instances that provide a baseline level of CPU performance with the ability to burst above the baseline.
    • T2 instances accumulate CPU Credits when they are idle, and consume CPU Credits when they are active.
    • T2 Unlimited Instances can sustain high CPU performance for as long as a workload needs it at an additional cost.
  • R for applications needing more RAM or Memory
  • C for applications needing more Compute
  • M for applications needing more Medium or Moderate performance on both Memory and CPU
  • I for applications needing more IOPS
  • G for applications needing more GPU

Instance Purchasing Option

  • On-Demand Instances
    • pay for instances and compute capacity that you use by the hour
    • no long-term commitments or up-front payments
  • Reserved Instances
    • provides lower hourly running costs by providing a billing discount
    • capacity reservation is applied to instances
    • suited if consistent, heavy, predictable usage
    • provides benefits with Consolidate Billing
    • can be modified to switch Availability Zones or the instance size within the same instance type, given the instance size footprint (Normalization factor) remains the same
    • pay for the entire term regardless of the usage
    • is not a physical instance that is launched, but rather a billing discount applied to the use of On-Demand Instances
  • Scheduled Reserved Instances
    • enable capacity reservations purchase that recurs on a daily, weekly, or monthly basis, with a specified start time and duration, for a one-year term.
    • Charges are incurred for the time that the instances are scheduled, even if they are not used
    • good choice for workloads that do not run continuously, but do run on a regular schedule
  • Spot Instances
    • cost-effective choice but does NOT guarantee availability
    • applications flexible in the timing when they can run and also able to handle interruption by storing the state externally
    • provides a two-minute warning if the instance is to be terminated to save any unsaved work
    • Spot blocks can also be launched with a required duration, which are not interrupted due to changes in the Spot price
    • Spot Fleet is a collection, or fleet, of Spot Instances, and optionally On-Demand Instances, which attempts to launch the number of Spot and On-Demand Instances to meet the specified target capacity
  • Dedicated Instances
    • is a tenancy option that enables instances to run in VPC on hardware that’s isolated, dedicated to a single customer
  • Dedicated Host
    • is a physical server with EC2 instance capacity fully dedicated to your use
  • Light, Medium, and Heavy Utilization Reserved Instances are no longer available for purchase and were part of the Previous Generation AWS EC2 purchasing model

Enhanced Networking

  • results in higher bandwidth, higher packet per second (PPS) performance, lower latency, consistency, scalability, and lower jitter
  • supported using Single Root – I/O Virtualization (SR-IOV) only on supported instance types
  • is supported only with a VPC (not EC2 Classic), HVM virtualization type and available by default on Amazon AMI but can be installed on other AMIs as well

Placement Group

  • Cluster Placement Group
    • provide low latency, High-Performance Computing via 10Gbps network
    • is a logical grouping on instances within a Single AZ
    • don’t span availability zones, can span multiple subnets but subnets must be in the same AZ
    • can span across peered VPCs for the same Availability Zones
    • existing instances can’t be moved into an existing placement group
    • An existing instance can be moved to a placement group, or moved from one placement group to another, or removed from a placement group, given it is in the stopped state.
    • for capacity errors, stop and start the instances in the placement group
    • use homogenous instance types which support enhanced networking and launch all the instances at once
  • Spread Placement Groups
    • is a group of instances that are each placed on distinct underlying hardware i.e. each instance on a distinct rack across AZ
    • recommended for applications that have a small number of critical instances that should be kept separate from each other.
    • reduces the risk of simultaneous failures that might occur when instances share the same underlying hardware.
  • Partition Placement Groups
    • is a group of instances spread across partitions i.e. group of instances spread across racks across AZs
    • reduces the likelihood of correlated hardware failures for the application.
    • can be used to spread deployment of large distributed and replicated workloads, such as HDFS, HBase, and Cassandra, across distinct hardware

EC2 Monitoring

  • CloudWatch provides monitoring for EC2 instances
  • Status monitoring helps quickly determine whether EC2 has detected any problems that might prevent instances from running applications.
  • Status monitoring includes
    • System Status checks – indicate issues with the underlying hardware
    • Instance Status checks – indicate issues with the underlying instance.

Elastic Load Balancer

  • Managed load balancing service and scales automatically
  • distributes incoming application traffic across multiple EC2 instances
  • is distributed system that is fault tolerant and actively monitored by AWS scales it as per the demand
  • are engineered to not be a single point of failure
  • need to Pre-Warm ELB if the demand is expected to shoot especially during load testing. AWS documentation does not mention it now.
  • supports routing traffic to instances in multiple AZs in the same region
  • performs Health Checks to route traffic only to the healthy instances
  • support Listeners with HTTP, HTTPS, SSL, TCP protocols
  • has an associated IPv4 and dual stack DNS name
  • can offload the work of encryption and decryption (SSL termination) so that the EC2 instances can focus on their main work
  • supports Cross Zone load balancing to help route traffic evenly across all EC2 instances regardless of the AZs they reside in
  • to help identify the IP address of a client
    • supports Proxy Protocol header for TCP/SSL connections
    • supports X-Forward headers for HTTP/HTTPS connections
  • supports Stick Sessions (session affinity) to bind a user’s session to a specific application instance,
    • it is not fault tolerant, if an instance is lost the information is lost
    • requires HTTP/HTTPS listener and does not work with TCP
    • requires SSL termination on ELB as it users the headers
  • supports Connection draining to help complete the in-flight requests in case an instance is deregistered
  • For High Availability, it is recommended to attach one subnet per AZ for at least two AZs, even if the instances are in a single subnet.
  • supports Static/Elastic IP (NLB only)
  • IPv4 & IPv6 support however VPC does not support IPv6. VPC now supports IPV6.
  • HTTPS listener does not support Client Side Certificate
  • For SSL termination at backend instances or support for Client Side Certificate use TCP for connections from the client to the ELB, use the SSL protocol for connections from the ELB to the back-end application, and deploy certificates on the back-end instances handling requests
  • supports a single SSL certificate, so for multiple SSL certificate multiple ELBs need to be created
  • Uses Server Name Indication to supports multiple SSL certificates

Application Load Balancer

  • supports HTTP and HTTPS (Secure HTTP) protocols
  • supports HTTP/2, which is enabled natively. Clients that support HTTP/2 can connect over TLS
  • supports WebSockets and Secure WebSockets natively
  • supports Request tracing, by default.
    • request tracing can be used to track HTTP requests from clients to targets or other services.
    • Load balancer upon receiving a request from a client, adds or updates the X-Amzn-Trace-Id header before sending the request to the target
  • supports containerized applications. Using Dynamic port mapping, ECS can select an unused port when scheduling a task and register the task with a target group using this port.
  • supports Sticky Sessions (Session Affinity) using load balancer generated cookies, to route requests from the same client to the same target
  • supports SSL termination, to decrypt the request on ALB before sending it to the underlying targets.
  • supports layer 7 specific features like X-Forwarded-For headers to help determine the actual client IP, port and protocol
  • automatically scales its request handling capacity in response to incoming application traffic.
  • supports hybrid load balancing, to route traffic to instances in VPC and an on-premises location
  • provides High Availability, by allowing more than one AZ to be specified
  • integrates with ACM to provision and bind a SSL/TLS certificate to the load balancer thereby making the entire SSL offload process very easy
  • supports multiple certificates for the same domain to a secure listener
  • supports IPv6 addressing, for an Internet facing load balancer
  • supports Cross-zone load balancing, and cannot be disabled.
  • supports Security Groups to control the traffic allowed to and from the load balancer.
  • provides Access Logs, to record all requests sent the load balancer, and store the logs in S3 for later analysis in compressed format
  • provides Delete Protection, to prevent the ALB from accidental deletion
  • supports Connection Idle Timeout – ALB maintains two connections for each request one with the Client (front end) and one with the target instance (back end). If no data has been sent or received by the time that the idle timeout period elapses, ALB closes the front-end connection
  • integrates with CloudWatch to provide metrics such as request counts, error counts, error types, and request latency
  • integrates with AWS WAF, a web application firewall that helps protect web applications from attacks by allowing rules configuration based on IP addresses, HTTP headers, and custom URI strings
  • integrates with CloudTrail to receive a history of ALB API calls made on the AWS account
  • back-end server authentication is NOT supported
  • does not provide Static, Elastic IP addresses

Network Load Balancer

  • handles volatile workloads and scale to millions of requests per second, without the need of pre-warming
  • offers extremely low latencies for latency-sensitive applications.
  • provides static IP/Elastic IP addresses for the load balancer
  • allows registering targets by IP address, including targets outside the VPC (on-premises) for the load balancer.
  • supports containerized applications. Using Dynamic port mapping, ECS can select an unused port when scheduling a task and register the task with a target group using this port.
  • monitors the health of its registered targets and routes the traffic only to healthy targets
  • enable cross-zone loading balancing only after creating the NLB
  • preserves client side source IP allowing the back-end to see client IP address. Target groups can be created with target type as instance ID or IP address. If targets registered by instance ID, the source IP addresses of the clients are preserved and provided to the applications. If register targets registered by IP address, the source IP addresses are the private IP addresses of the load balancer nodes.
  • supports both network and application target health checks.
  • supports long-lived TCP connections ideal for WebSocket type of applications
  • supports Zonal Isolation, which is designed for application architectures in a single zone and can be enabled in a single AZ to support architectures that require zonal isolation
  • does not support stick sessions

Auto Scaling

  • ensures correct number of EC2 instances are always running to handle the load by scaling up or down automatically as demand changes
  • cannot span multiple regions.
  • attempts to distribute instances evenly between the AZs that are enabled for the Auto Scaling group
  • performs checks either using EC2 status checks or can use ELB health checks to determine the health of an instance and terminates the instance if unhealthy, to launch a new instance
  • can be scaled using manual scaling, scheduled scaling or demand based scaling
  • cooldown period helps ensure instances are not launched or terminated before the previous scaling activity takes effect to allow the newly launched instances to start handling traffic and reduce load

AWS Auto Scaling & ELB

  • Auto Scaling & ELB can be used for High Availability and Redundancy by spanning Auto Scaling groups across multiple AZs within a region and then setting up ELB to distribute incoming traffic across those AZs
  • With Auto Scaling, use ELB health check with the instances to ensure that traffic is routed only to the healthy instances

Lambda

  • offers Serverless computing that allows applications and services to be built and run without thinking about servers.
  • helps run code without provisioning or managing servers, where you pay only for the compute time when the code is running.
  • is priced on a pay-per-use basis and there are no charges when the code is not running.
  • performs all the operational and administrative activities on your behalf, including capacity provisioning, monitoring fleet health, applying security patches to the underlying compute resources, deploying code, running a web service front end, and monitoring and logging the code.
  • does not provide access to the underlying compute infrastructure.
  • handles scalability and availability as it
    • provides easy scaling and high availability to the code without additional effort on your part.
    • is designed to process events within milliseconds.
    • is designed to run many instances of the functions in parallel.
    • is designed to use replication and redundancy to provide high availability for both the service and the functions it operates.
    • has no maintenance windows or scheduled downtimes for either.
    • has a default safety throttle for the number of concurrent executions per account per region.
    • has a higher latency immediately after a function is created, or updated, or if it has not been used recently.
    • for any function updates, there is a brief window of time, less than a minute, when requests would be served by both versions
  • Security
    • stores code in S3 and encrypts it at rest and performs additional integrity checks while the code is in use.
    • each function runs in its own isolated environment, with its own resources and file system view
    • supports Code Signing using AWS Signer, which offers trust and integrity controls that enable you to verify that only unaltered code from approved developers is deployed in the functions.
  • Functions must complete execution within 900 seconds. The default timeout is 3 seconds. The timeout can be set the timeout to any value between 1 and 900 seconds.
  • AWS Step Functions can help coordinate a series of Lambda functions in a specific order. Multiple functions can be invoked sequentially, passing the output of one to the other, and/or in parallel, while the state is being maintained by Step Functions.
  • AWS X-Ray helps to trace functions, which provides insights such as service overhead, function init time, and function execution time.
  • Lambda Provisioned Concurrency provides greater control over the performance of serverless applications.
  • Lambda@Edge allows you to run code across AWS locations globally without provisioning or managing servers, responding to end-users at the lowest network latency.
  • Lambda Extensions allow integration of Lambda with other third-party tools for monitoring, observability, security, and governance.
  • Compute Savings Plan can help save money for Lambda executions.
  • CodePipeline and CodeDeploy can be used to automate the serverless application release process.
  • RDS Proxy provides a highly available database proxy that manages thousands of concurrent connections to relational databases.
  • Supports Elastic File Store, to provide a shared, external, persistent, scalable volume using a fully managed elastic NFS file system without the need for provisioning or capacity management.
  • Supports Function URLs, a built-in HTTPS endpoint that can be invoked using the browser, curl, and any HTTP client.