AWS SQS – Standard vs FIFO Queue – Certification

AWS SQS – Standard vs FIFO Queue

SQS offers two types of queues – Standard & FIFO queues

SQS Standard vs FIFO Queues

Message Order

  • Standard queues provide best-effort ordering which ensures that messages are generally delivered in the same order as they are sent. Occasionally (because of the highly-distributed architecture that allows high throughput), more than one copy of a message might be delivered out of order
  • FIFO queues offer first-in-first-out delivery and exactly-once processing: the order in which messages are sent and received is strictly preserved

Delivery

  • Standard queues guarantee that a message is delivered at least once and duplicates can be introduced into the queue
  • FIFO queues ensure a message is delivered exactly once and remains available until a consumer processes and deletes it; duplicates are not introduced into the queue

Transactions Per Second (TPS)

  • Standard queues allow nearly-unlimited number of transactions per second
  • FIFO queues are limited to 300 transactions per second per API action

Regions

  • Standard queues are available in all the regions
  • FIFO queues are currently available in limited regions including the US West (Oregon), US East (Ohio), US East (N. Virginia), and EU (Ireland)

SQS Buffered Asynchronous Client

  • FIFO queues aren’t currently compatible with the SQS Buffered Asynchronous Client, where messages are buffered at client side and send as a single request to the SQS queue to reduce cost.

AWS Services Supported

  • Standard Queues are supported by all AWS services
  • FIFO Queues are currently not supported by all AWS services like
    • CloudWatch Events
    • S3 Event Notifications
    • SNS Topic Subscriptions
    • Auto Scaling Lifecycle Hooks
    • AWS IoT Rule Actions
    • AWS Lambda Dead Letter Queues

Use Cases

  • Standard queues can be used in any scenarios, as long as the application can process messages that arrive more than once and out of order
    • Decouple live user requests from intensive background work: Let users upload media while resizing or encoding it.
    • Allocate tasks to multiple worker nodes: Process a high number of credit card validation requests.
    • Batch messages for future processing: Schedule multiple entries to be added to a database.
  • FIFO queues are designed to enhance messaging between applications when the order of operations and events is critical, or where duplicates can’t be tolerated
    • Ensure that user-entered commands are executed in the right order.
    • Display the correct product price by sending price modifications in the right order.
    • Prevent a student from enrolling in a course before registering for an account.

[do_widget id=text-15]

AWS Certification Exam Practice Questions

  • Questions are collected from Internet and the answers are marked as per my knowledge and understanding (which might differ with yours).
  • AWS services are updated everyday and both the answers and questions might be outdated soon, so research accordingly.
  • AWS exam questions are not updated to keep up the pace with AWS updates, so even if the underlying feature has changed the question might not be updated
  • Open to further feedback, discussion and correction.
  1. A restaurant reservation application needs the ability to maintain a waiting list. When a customer tries to reserve a table, and none are available, the customer must be put on the waiting list, and the application must notify the customer when a table becomes free. What service should the Solutions Architect recommend ensuring that the system respects the order in which the customer requests are put onto the waiting list?
    1. Amazon SNS
    2. AWS Lambda with sequential dispatch
    3. A FIFO queue in Amazon SQS
    4. A standard queue in Amazon SQS
  2. A solutions architect is designing an application for a two-step order process. The first step is synchronous and must return to the user with little latency. The second step takes longer, so it will be implemented in a separate component. Orders must be processed exactly once and in the order in which they are received. How should the solutions architect integrate these components?
    1. Use Amazon SQS FIFO queues.
    2. Use an AWS Lambda function along with Amazon SQS standard queues.
    3. Create an SNS topic and subscribe an Amazon SQS FIFO queue to that topic.
    4. Create an SNS topic and subscribe an Amazon SQS Standard queue to that topic.

AWS SQS FIFO Queue

AWS SQS FIFO Queue

  • SQS FIFO Queue provides enhanced messaging between applications with the additional features
    • FIFO (First-In-First-Out) delivery
      • order in which messages are sent and received is strictly preserved
      • key when the order of operations & events is critical
    • Exactly-once processing
      • a message is delivered once and remains available until a consumer processes and deletes it
      • key when duplicates can’t be tolerated.
      • limited to 300 or 3000 (with batching) transactions per second (TPS)
  • FIFO queues provide all the capabilities as Standard queues and improves upon and complements the standard queue.
  • FIFO queues support message groups that allow multiple ordered message groups within a single queue.
  • FIFO queues are available in the US East (N. Virginia), US East (Ohio), US West (Oregon), and EU (Ireland) regions (keeps on changing)
  • FIFO Queue name should end with .fifo
  • SQS Buffered Asynchronous Client doesn’t currently support FIFO queues
  • Not all the AWS Services support FIFO like
    • Auto Scaling Lifecycle Hooks
    • Amazon CloudWatch Events
    • AWS IoT Rule Actions
    • AWS Lambda Dead-Letter Queues
    • Amazon S3 Event Notifications
    • Amazon SNS Topic Subscriptions
  • SQS FIFO supports one or more producers and messages are stored in the order that they were successfully received by SQS.
  • SQS FIFO queues don’t serve messages from the same message group to more than one consumer at a time.

Message Deduplication

  • SQS APIs provide deduplication functionality that prevents message producer from sending duplicates.
  • Message deduplication ID is the token used for deduplication of sent messages.
  • If a message with a particular message deduplication ID is sent successfully, any messages sent with the same message deduplication ID are accepted successfully but aren’t delivered during the 5-minute deduplication interval.
  • So basically, any duplicates introduced by the message producer are removed within a 5-minute deduplication interval
  • Message deduplication applies to an entire queue, not to individual message groups

Message groups

  • Messages are grouped into distinct, ordered “bundles” within a FIFO queue
  • Message group ID is the tag that specifies that a message belongs to a specific message group
  • For each message group ID, all messages are sent and received in strict order
  • However, messages with different message group ID values might be sent and received out of order.
  • Every message must be associated with a message group ID, without which the action fails.
  • SQS delivers the messages in the order in which they arrive for processing, if multiple hosts (or different threads on the same host) send messages with the same message group ID

SQS Standard Queues vs SQS FIFO Queues

Refer Blog Post – SQS Standard Queues vs FIFO Queues

[do_widget id=text-15]

AWS Certification Exam Practice Questions

  • Questions are collected from Internet and the answers are marked as per my knowledge and understanding (which might differ with yours).
  • AWS services are updated everyday and both the answers and questions might be outdated soon, so research accordingly.
  • AWS exam questions are not updated to keep up the pace with AWS updates, so even if the underlying feature has changed the question might not be updated
  • Open to further feedback, discussion and correction.
  1. A restaurant reservation application needs the ability to maintain a waiting list. When a customer tries to reserve a table, and none are available, the customer must be put on the waiting list, and the application must notify the customer when a table becomes free. What service should the Solutions Architect recommend ensuring that the system respects the order in which the customer requests are put onto the waiting list?
    1. Amazon SNS
    2. AWS Lambda with sequential dispatch
    3. A FIFO queue in Amazon SQS
    4. A standard queue in Amazon SQS
  2. In relation to Amazon SQS, how can you ensure that messages are delivered in order? Select 2 answers
    1. Increase the size of your queue
    2. Send them with a timestamp
    3. Using FIFO queues
    4. Give each message a unique id
    5. Use sequence number within the messages with Standard queues
  3. A company has run a major auction platform where people buy and sell a wide range of products. The platform requires that transactions from buyers and sellers get processed in exactly the order received. At the moment, the platform is implemented using RabbitMQ, which is a light weighted queue system. The company consulted you to migrate the on-premise platform to AWS. How should you design the migration plan? (Select TWO)
    1. When the bids are received, send the bids to a SQS FIFO queue before they are processed.
    2. When the users have submitted the bids from frontend, the backend service delivers the messages to a SQS standard queue.
    3. Add a message group ID to the messages before they are sent to the SQS queue so that the message processing is in a strict order.  
    4. Use an EC2 or Lambda to add a deduplication ID to the messages before the messages are sent to the SQS queue to ensure that bids are processed in the right order.