AWS Lambda

AWS Lambda

  • AWS Lambda offers Serverless computing
  • Serverless computing allows applications and services to be built and run without thinking about servers. With serverless computing, application still runs on servers, but all the server management is done by AWS.
  • Lambda lets you run code without provisioning or managing servers, where you pay only for the compute time when the code is running.
  • Lambda is priced on a pay-per-use basis and there are no charges when the code is not running.
  • Lambda allows the running of code for any type of application or backend service with zero administration.
  • Lambda performs all the operational and administrative activities on your behalf, including capacity provisioning, monitoring fleet health, applying security patches to the underlying compute resources, deploying code, running a web service front end, and monitoring and logging the code.
  • Lambda does not provide access to the underlying compute infrastructure.
  • Lambda handles Scalability and availability as it
    • provides easy scaling and high availability to the code without additional effort on your part.
    • is designed to process events within milliseconds.
    • is designed to run many instances of the functions in parallel.
    • is designed to use replication and redundancy to provide high availability for both the service and the functions it operates.
    • has no maintenance windows or scheduled downtimes for either.
    • has a default safety throttle for the number of concurrent executions per account per region.
    • has a higher latency immediately after a function is created, or updated, or if it has not been used recently.
    • for any function updates, there is a brief window of time, less than a minute, when requests would be served by both versions
  • Security
    • stores code in S3 and encrypts it at rest and performs additional integrity checks while the code is in use.
    • each function runs in its own isolated environment, with its own resources and file system view
  • AWS Lambda functions must complete execution within 300 900 seconds. The default timeout is 3 seconds. The timeout can be set the timeout to any value between 1 and 300 900 seconds.
  • AWS Step Functions can help coordinate a series of Lambda functions in a specific order. Multiple Lambda functions can be invoked sequentially, passing the output of one to the other, and/or in parallel, while the state is being maintained by Step Functions.
  • AWS X-Ray helps tracing for Lambda functions, which provides insights such as service overhead, function init time, and function execution time.

Functions & Event Sources

Core components of Lambda are functions and event sources.

  • Event source is an AWS service or custom application that publishes events.
  • A function is the custom code that processes the events

Lambda Functions

  • Each function has associated configuration information, such as its name, description, entry point, and resource requirements
  • Lambda may choose to retain an instance of the function and reuse it to serve a subsequent request, rather than creating a new copy.
  • Each function receives 500MB of non-persistent disk space in its own /tmp directory.
  • Design Lambda function as stateless
    • functions should be stateless, to allow launching as many copies of the function as needed as per the demand.
    • Local file system access, child processes, and similar artifacts may not extend beyond the lifetime of the request
    • The state can be maintained externally in DynamoDB or S3
  • A function can be granted permissions to access other resources using an IAM role
  • Functions have the following restrictions
    • Inbound network connections are blocked
    • Outbound connections only TCP/IP sockets are supported
    • ptrace (debugging) system calls are blocked
    • TCP port 25 traffic is also blocked as an anti-spam measure.
  • Functions are automatically monitored, real-time metrics reported through CloudWatch, including total requests, latency, error rates, and throttled requests.
  • Lambda automatically integrates with CloudWatch logs, creating a log group for each function and providing basic application lifecycle event log entries, including logging the resources consumed for each use of that function.
  • Functions support code written in
    • Node.js (JavaScript)
    • Python
    • Ruby
    • Java (Java 8 compatible)
    • C# (.NET Core)
    • Go
    • Custom runtime
  • Failure Handling
    • For S3 bucket notifications and custom events, Lambda will attempt execution of the function three times in the event of an error condition in the code or if a service or resource limit is exceeded
    • For ordered event sources that Lambda polls, e.g. DynamoDB Streams and Kinesis streams, it will continue attempting execution in the event of a developer code error until the data expires.
    • Kinesis and DynamoDB Streams retain data for a minimum of 24 hours
    • Dead Letter Queues can be configured for events to be placed, once the retry policy for asynchronous invocations is exceeded

Lambda Function Limits

  • RAM – 128 MB to 10,240 MB (10 GB)
  • CPU is linked to RAM and cannot be set manually.
    • 2 vCPUs = 1769 MB RAM
    • 6 vCPUs = 10240 MB RAM
  • Timeout – 900 Secs or 15 mins
  • /tmp storage – 512 MB
  • Deployment Package – 50 MB (zipped), 250 MB (unzipped) including layers
  • Cocurrent Executions – 1000 (soft limit)
  • Container Image Size – 10 GB
  • Invocation Payload (request/response) – 6 MB (sync), 256 KB (async)

Lambda Function Versions

  • Function versions can be used to manage the deployment of the functions.
  • Each function has a single, current version of the code and there is no versioning of the same function.
  • Lambda creates a new version of the function each time it’s published. The new version is a copy of the unpublished version of the function.
  • A function version includes the following information:
    • The function code and all associated dependencies.
    • The Lambda runtime that invokes the function.
    • All the function settings, including the environment variables.
    • A unique Amazon Resource Name (ARN) to identify the specific version of the function.
  • Lambda supports creating aliases, which are mutable, for each Lambda function version.
  • Alias is a pointer to a specific function version, with a unique ARN.
  • Each alias maintains an ARN for a function version to which it points.
  • An alias can only point to a function version, not to another alias
  • Alias helps in rolling out new changes or rolling back to old versions
  • Alias supports routing configuration to point to a maximum of two Lambda function versions. It can be used for canary testing to send a portion of traffic to a second function version.

Lambda Event Sources

Refer Blog Post – Lambda Event Sources

Lambda Execution Environment

  • Lambda invokes the function in an execution environment, which provides a secure and isolated runtime environment.
  • Execution environment takes care of provisioning and managing the resources needed to run the function.
  • Execution environment also provides lifecycle support for the function’s runtime and any external extensions associated with the function.
  • Function’s runtime communicates with Lambda using the Runtime API.
  • Extensions communicate with Lambda using the Extensions API.
  • Extensions can also receive log messages from the function by subscribing to logs using the Logs API.
  • Execution Context is a temporary runtime environment that initializes any external dependencies of the Lambda function code, for e.g. database connections or HTTP endpoints.
  • When a function is invoked, the Execution environment is launched based on the provided configuration settings i.e. memory and execution time.
  • After a Lambda function is executed, Lambda maintains the execution environment for some time in anticipation of another function invocation which allows it to reuse the /tmp directory and objects declared outside of the function’s handler method e.g. database connection.
  • When a Lambda function is invoked for the first time or after it has been updated there is latency for bootstrapping as Lambda tries to reuse the Execution Context for subsequent invocations of the Lambda function
  • Subsequent invocations perform better performance as there is no need to “cold-start” or initialize those external dependencies
  • Lambda manages Execution Environment creations and deletion, there is no AWS Lambda API to manage Execution Environment.

Lambda Execution Environment

Lambda in VPC

  • Lambda function always runs inside a VPC owned by the Lambda service which isn’t connected to your account’s default VPC
  • Lambda applies network access and security rules to this VPC and maintains and monitors the VPC automatically.
  • A function can be configured to connect to private subnets in a VPC in the AWS account.
  • Function connected to VPC can access private resources databases, cache instances, or internal services during the execution.
  • To enable the function to access resources inside the private VPC, additional VPC-specific configuration information that includes private subnet IDs and security group IDs must be provided.
  • Lambda uses this information to set up ENIs that enables the function to connect securely to other resources within your private VPC.
  • Functions connected to VPC can’t access the Internet and need a NAT Gateway to access any external resources outside of AWS.
  • Functions cannot connect directly to a VPC with dedicated instance tenancy, instead, peer it to a second VPC with default tenancy.

Lambda Security

  • All data stored in ephemeral storage is encrypted at rest with a key managed by AWS.
  • Lambda functions provide access only to a single VPC. If multiple subnets are specified, they must all be in the same VPC. Other VPCs can be connected using VPC Peering.
  • Supports Code Signing which offers trust and integrity controls that enable you to verify that only unaltered code from approved developers is deployed in the functions. 
  • AWS Lambda can perform the following signature checks at deployment:
    • Corrupt signature – This occurs if the code artifact has been altered since signing.
    • Mismatched signature – This occurs if the code artifact is signed by a signing profile that is not approved.
    • Expired signature – This occurs if the signature is past the configured expiry date.
    • Revoked signature – This occurs if the signing profile owner revokes the signing jobs.
  • For sensitive information, for e.g. passwords, AWS recommends using client-side encryption using AWS Key Management Service and store the resulting values as ciphertext in your environment variable.
  • Function code should include the logic to decrypt these values.

Lambda Permissions

  • IAM – Use IAM to manage access to the Lambda API and resources like functions and layers.
  • Execution Role – A Lambda function can be provided with an Execution Role, that grants it permission to access AWS services and resources e.g. send logs to CloudWatch and upload trace data to AWS X-Ray.
  • Resource-based Policies
    • Use resource-based policies to give other accounts and AWS services permission to use the Lambda resources.
    • Resource-based permissions policies are supported for functions and layers.

Invoking Lambda Functions

  • Lambda functions can be invoked
    • directly using the Lambda console or API, a function URL HTTP(S) endpoint, an AWS SDK, the AWS CLI, and AWS toolkits.
    • other AWS services like S3 and SNS invoke the function.
    • to read from a stream or queue and invoke the function.
  • Functions can be invoked
    • Synchronously
      • You wait for the function to process the event and return a response.
      • Error handling and retries need to be handled by the Client.
      • Invocation includes API, and SDK for called from API Gateway.
    • Asynchronously
      • queues the event for processing and returns a response immediately.
      • handles retries and can send invocation records to a destination for successful and failed events.
      • Invocation includes S3, SNS, and CloudWatch Events
      • can define DLQ for handling failed events. AWS recommends using destination instead of DLQ.

Lambda Provisioned Concurrency

  • Provisioned Concurrency provides greater control over the performance of the serverless applications.
  • When enabled, Provisioned Concurrency keeps functions initialized and hyper-ready to respond in double-digit milliseconds.
  • Provisioned Concurrency is ideal for building latency-sensitive applications, such as web or mobile backends, synchronously invoked APIs and interactive microservices.
  • The amount of concurrency can be increased during times of high demand and lowered or turn it off completely, when demand decreases.
  • If the concurrency of a function reaches the configured level, subsequent invocations of the function have the latency and scale characteristics of regular Lambda functions.

Lambda@Edge

Refer blog post @ Lambda@Edge

Lambda Best Practices

  • Lambda function code should be stateless and ensure there is no affinity between the code and the underlying compute infrastructure.
  • Instantiate AWS clients outside the scope of the handler to take advantage of connection re-use.
  • Make sure you have set +rx permissions on your files in the uploaded ZIP to ensure Lambda can execute code on your behalf.
  • Lower costs and improve performance by minimizing the use of startup code not directly related to processing the current event.
  • Use the built-in CloudWatch monitoring of the Lambda functions to view and optimize request latencies.
  • Delete old Lambda functions that you are no longer using.

AWS Certification Exam Practice Questions

  • Questions are collected from Internet and the answers are marked as per my knowledge and understanding (which might differ with yours).
  • AWS services are updated everyday and both the answers and questions might be outdated soon, so research accordingly.
  • AWS exam questions are not updated to keep up the pace with AWS updates, so even if the underlying feature has changed the question might not be updated
  • Open to further feedback, discussion and correction.
  • Questions are collected from Internet and the answers are marked as per my knowledge and understanding (which might differ with yours).
  • AWS services are updated everyday and both the answers and questions might be outdated soon, so research accordingly.
  • AWS exam questions are not updated to keep up the pace with AWS updates, so even if the underlying feature has changed the question might not be updated
  • Open to further feedback, discussion and correction.
  1. Your serverless architecture using AWS API Gateway, AWS Lambda, and AWS DynamoDB experienced a large increase in traffic to a sustained 400 requests per second, and dramatically increased in failure rates. Your requests, during normal operation, last 500 milliseconds on average. Your DynamoDB table did not exceed 50% of provisioned throughput, and Table primary keys are designed correctly. What is the most likely issue?
    1. Your API Gateway deployment is throttling your requests.
    2. Your AWS API Gateway Deployment is bottlenecking on request (de)serialization.
    3. You did not request a limit increase on concurrent Lambda function executions. (Refer link – AWS API Gateway by default throttles at 500 requests per second steady-state, and 1000 requests per second at spike. Lambda, by default, throttles at 100 concurrent requests for safety. At 500 milliseconds (half of a second) per request, you can expect to support 200 requests per second at 100 concurrency. This is less than the 400 requests per second your system now requires. Make a limit increase request via the AWS Support Console.)
    4. You used Consistent Read requests on DynamoDB and are experiencing semaphore lock.

AWS Lambda Event Source

AWS Lambda Event Source

  • Core components of Lambda are Lambda functions and event sources.
    • An AWS Lambda event source is the AWS service or custom application that publishes events
    • Lambda function is the custom code that processes the events
  • An event source is an AWS service or developer-created application that produces events that trigger an AWS Lambda function to run
  • Event sources can be either AWS Services or Custom applications
  • Event sources can be both push and pull sources
    • Services like S3, and SNS publish events to Lambda by invoking the cloud function directly
    • Lambda can also poll resources in services like Kafka, and Kinesis streams that do not publish events to Lambda
  • Events are passed to a Lambda function as an event input parameter. For batch event sources, such as Kinesis Streams, the event parameter may contain multiple events in a single call, based on the requested batch size

Lambda Event Source Mapping

  • Lambda Event source mapping refers to the configuration which maps an event source to a Lambda function.
  • Event source mapping enables automatic invocation of the Lambda function when events occur.
  • Each event source mapping identifies the type of events to publish and the Lambda function to invoke when events occur
  • AWS supported event sources can be grouped into
    • Regular AWS services
      • also referred to as the Push model
      • includes services like S3, SNS, SES etc.
      • event source mapping maintained on their side
      • as the event sources invoke the Lambda function, a resource-based policy should be used to grant the event source necessary permissions
    • Stream-based event sources
      • also referred to as the Pull model
      • includes services like DynamoDB, Kinesis streams, MQ, SQS, Kafka
      • need to have the event source mapping maintained on the Lambda side
  • Functions can be invoked
    • Synchronously
      • You wait for the function to process the event and return a response.
      • Error handling and retries need to be handled by the Client.
      • Invocation includes API, and SDK for called from API Gateway.
    • Asynchronously
      • queues the event for processing and returns a response immediately.
      • handles retries and can send invocation records to a destination for successful and failed events.
      • Invocation includes S3, SNS, and CloudWatch Events

Lambda Supported Event Sources

AWS Lambda can be configured as an event source for multiple AWS services

Service Method of invocation
Amazon Alexa Event-driven; synchronous invocation
Amazon MSK – Managed Streaming for Apache Kafka Lambda polling
Self-managed Apache Kafka Lambda polling
Amazon API Gateway Event-driven; synchronous invocation
AWS CloudFormation Event-driven; asynchronous invocation
Amazon CloudFront (Lambda@Edge) Event-driven; synchronous invocation
Amazon EventBridge (CloudWatch Events) Event-driven; asynchronous invocation
Amazon CloudWatch Logs Event-driven; asynchronous invocation
AWS CodeCommit Event-driven; asynchronous invocation
AWS CodePipeline Event-driven; asynchronous invocation
Amazon Cognito Event-driven; synchronous invocation
AWS Config Event-driven; asynchronous invocation
Amazon Connect Event-driven; synchronous invocation
Amazon DynamoDB Lambda polling
Amazon Elastic File System Special integration
Elastic Load Balancing (Application Load Balancer) Event-driven; synchronous invocation
AWS IoT Event-driven; asynchronous invocation
AWS IoT Events Event-driven; asynchronous invocation
Amazon Kinesis Lambda polling
Amazon Kinesis Data Firehose Event-driven; synchronous invocation
Amazon Lex Event-driven; synchronous invocation
Amazon MQ Lambda polling
Amazon Simple Email Service Event-driven; asynchronous invocation
Amazon Simple Notification Service Event-driven; asynchronous invocation
Amazon Simple Queue Service Lambda polling
Amazon S3 Event-driven; asynchronous invocation
Amazon Simple Storage Service Batch Event-driven; synchronous invocation
Secrets Manager Event-driven; synchronous invocation
AWS X-Ray Special integration

Amazon S3

  • S3 bucket events, such as the object-created or object-deleted events can be processed using Lambda functions for e.g., Lambda function can be invoke when a user uploads a photo to a bucket to read the image and create a thumbnail
  • S3 bucket notification configuration feature can be configured for the event source mapping, to identify the S3 bucket events and the Lambda function to invoke.
  • Error handling for an event source depends on how Lambda is invoked
  • S3 invokes your Lambda function asynchronously.

DynamoDB

  • Lambda functions can be used as triggers for the DynamoDB table to take custom actions in response to updates made to the DynamoDB table.
  • Trigger can be created by
    • First enabling DynamoDB Streams for the table.
    • Lambda polls the stream and processes any updates published to the stream
  • DynamoDB is a stream-based event source and with stream-based service, the event source mapping is created in Lambda, identifying the stream to poll and which Lambda function to invoke.
  • Error handling for an event source depends on how Lambda is invoked

Kinesis Streams

  • AWS Lambda can be configured to automatically poll the Kinesis stream periodically (once per second) for new records
  • Lambda can process any new records such as social media feeds, IT logs, website click streams, financial transactions, and location-tracking events
  • Kinesis Streams is a stream-based event source and with stream-based service, the event source mapping is created in Lambda, identifying the stream to poll and which Lambda function to invoke.
  • Error handling for an event source depends on how Lambda is invoked

Simple Notification Service – SNS

  • SNS notifications can be processed using Lambda
  • When a message is published to an SNS topic, the service can invoke Lambda function by passing the message payload as parameter, which can then process the event
  • Lambda function can be triggered in response to CloudWatch alarms and other AWS services that use SNS.
  • SNS via topic subscription configuration feature can be used for the event source mapping, to identify the SNS topic and the Lambda function to invoke
  • Error handling for an event source depends on how Lambda is invoked
  • SNS invokes your Lambda function asynchronously.

Simple Email Service – SES

  • SES can be used to receive messages and can be configured to invoke Lambda function when messages arrive, by passing in the incoming email event as parameter
  • SES using the rule configuration feature can be used for the event source mapping
  • Error handling for an event source depends on how Lambda is invoked
  • SES invokes your Lambda function asynchronously.

Amazon Cognito

  • Cognito Events feature enables Lambda function to run in response to events in Cognito for e.g. Lambda function can be invoked for the Sync Trigger events, that is published each time a dataset is synchronized.
  • Cognito event subscription configuration feature can be used for the event source mapping
  • Error handling for an event source depends on how Lambda is invoked
  • Cognito is configured to invoke a Lambda function synchronously

CloudFormation

  • Lambda function can be specified as a custom resource to execute any custom commands as a part of deploying CloudFormation stacks and can be invoked whenever the stacks are created, updated or deleted.
  • CloudFormation using stack definition can be used for the event source mapping
  • Error handling for an event source depends on how Lambda is invoked
  • CloudFormation invokes the Lambda function asynchronously

CloudWatch Logs

  • Lambda functions can be used to perform custom analysis on CloudWatch Logs using CloudWatch Logs subscriptions.
  • CloudWatch Logs subscriptions provide access to a real-time feed of log events from CloudWatch Logs and deliver it to the AWS Lambda function for custom processing, analysis, or loading to other systems.
  • CloudWatch Logs using the log subscription configuration can be used for the event source mapping
  • Error handling for an event source depends on how Lambda is invoked
  • CloudWatch Logs invokes the Lambda function asynchronously

CloudWatch Events

  • CloudWatch Events help respond to state changes in the AWS resources. When the resources change state, they automatically send events into an event stream.
  • Rules that match selected events in the stream can be created to route them to the Lambda function to take action for e.g., the Lambda function can be invoked to log the state of an EC2 instance or AutoScaling Group.
  • CloudWatch Events by using a rule target definition can be used for the event source mapping
  • Error handling for an event source depends on how Lambda is invoked
  • CloudWatch Events invokes the Lambda function asynchronously

CodeCommit

  • Trigger can be created for a CodeCommit repository so that events in the repository will invoke a Lambda function for e.g., Lambda function can be invoked when a branch or tag is created or when a push is made to an existing branch.
  • CodeCommit by using a repository trigger can be used for the event source mapping
  • Error handling for an event source depends on how Lambda is invoked
  • CodeCommit Events invokes the Lambda function asynchronously

Scheduled Events (powered by CloudWatch Events)

  • AWS Lambda can be invoked regularly on a scheduled basis using the schedule event capability in CloudWatch Events.
  • CloudWatch Events by using a rule target definition can be used for the event source mapping
  • Error handling for an event source depends on how Lambda is invoked
  • CloudWatch Events invokes the Lambda function asynchronously

AWS Config

  • Lambda functions can be used to evaluate whether the AWS resource configurations comply with custom Config rules.
  • As resources are created, deleted, or changed, AWS Config records these changes and sends the information to the Lambda functions, which can then evaluate the changes and report results to AWS Config. AWS Config can be used to assess overall resource compliance
  • AWS Config by using a rule target definition can be used for the event source mapping
  • Error handling for an event source depends on how Lambda is invoked
  • AWS Config invokes the Lambda function asynchronously

Amazon API Gateway

  • Lambda function can be invoked over HTTPS by defining a custom REST API and endpoint using Amazon API Gateway.
  • Individual API operations, such as GET and PUT, can be mapped to specific Lambda functions.
  • When an HTTPS request to the API endpoint is received, the API Gateway service invokes the corresponding Lambda function.
  • Error handling for an event source depends on how Lambda is invoked.
  • API Gateway is configured to invoke a Lambda function synchronously.

Other Event Sources: Invoking a Lambda Function On Demand

  • Lambda functions can be invoked on-demand without the need to preconfigure any event source mapping in this case.

AWS Certification Exam Practice Questions

  • Questions are collected from Internet and the answers are marked as per my knowledge and understanding (which might differ with yours).
  • AWS services are updated everyday and both the answers and questions might be outdated soon, so research accordingly.
  • AWS exam questions are not updated to keep up the pace with AWS updates, so even if the underlying feature has changed the question might not be updated
  • Open to further feedback, discussion and correction.

References

AWS_Lambda_Developer_Guide