AWS SQS Standard vs FIFO Queue

SQS offers two types of queues – Standard & FIFO queues

SQS Standard vs FIFO Queue Features

Message Order

• Standard queues provide best-effort ordering which ensures that messages are generally delivered in the same order as they are sent. Occasionally (because of the highly-distributed architecture that allows high throughput), more than one copy of a message might be delivered out of order
• FIFO queues offer first-in-first-out delivery and exactly-once processing: the order in which messages are sent and received is strictly preserved

Delivery

• Standard queues guarantee that a message is delivered at least once and duplicates can be introduced into the queue
• FIFO queues ensure a message is delivered exactly once and remains available until a consumer processes and deletes it; duplicates are not introduced into the queue

Transactions Per Second (TPS)

• Standard queues allow nearly-unlimited number of transactions per second
• FIFO queues by default are limited to 300 transactions per second per API action (SendMessage, ReceiveMessage, DeleteMessage).
• With High Throughput Mode enabled, FIFO queues can support up to 70,000 TPS per API action without batching in select regions (US East N. Virginia, US West Oregon, Europe Ireland), and up to 700,000 messages per second with batching.
• High Throughput Mode can be enabled from the SQS console and uses message group-level partitioning to achieve higher throughput.
• In other regions, high throughput quotas vary (up to 18,000 TPS in several regions).

In-Flight Messages

• Standard queues support approximately 120,000 in-flight messages.
• FIFO queues now support up to 120,000 in-flight messages (increased from 20,000 in November 2024). In-flight messages are those received by a consumer but not yet deleted from the queue.

Regions

• Standard & FIFO queues are available in all regions where Amazon SQS is available.

SQS Buffered Asynchronous Client

• FIFO queues are not compatible with the SQS Buffered Asynchronous Client, where messages are buffered at the client side and sent as a single request to the SQS queue to reduce cost.

Dead-Letter Queue (DLQ) Support

• Both Standard and FIFO queues support dead-letter queues for handling messages that cannot be processed after a configured number of retries.
• FIFO queues now support DLQ redrive, allowing messages to be moved from a FIFO dead-letter queue back to the FIFO source queue or a custom FIFO destination queue (launched 2023, expanded to GovCloud in April 2024).

CloudWatch Metrics

• Standard queues support all standard SQS CloudWatch metrics.
• FIFO queues support additional metrics (added July 2024):
  • NumberOfDeduplicatedSentMessages – tracks deduplicated messages
  • ApproximateNumberOfGroupsWithInflightMessages – tracks active message groups

AWS Services Supported

• Standard Queues are supported by all AWS services
• FIFO Queues now have broader service integration than at launch, but some limitations remain:
  • Supported:
    • Amazon SNS FIFO Topics (can subscribe SQS FIFO queues for ordered fan-out)
    • AWS Lambda (SQS FIFO as event source mapping, with ordered processing per message group)
    • Amazon EventBridge (SQS FIFO as rule target; EventBridge Pipes supports FIFO as source)
    • Auto Scaling Lifecycle Hooks (SQS queue target)
  • Not Supported:
    • S3 Event Notifications (cannot directly target SQS FIFO; use EventBridge as intermediary)
    • Lambda Asynchronous Invocation Destinations (does not support SQS FIFO or SNS FIFO as destination)

Use Cases

• Standard queues can be used in any scenario, as long as the application can process messages that arrive more than once and out of order
  • Decouple live user requests from intensive background work: Let users upload media while resizing or encoding it.
  • Allocate tasks to multiple worker nodes: Process a high number of credit card validation requests.
  • Batch messages for future processing: Schedule multiple entries to be added to a database.
• FIFO queues are designed to enhance messaging between applications when the order of operations and events is critical, or where duplicates can’t be tolerated
  • Ensure that user-entered commands are executed in the right order.
  • Display the correct product price by sending price modifications in the right order.
  • Prevent a student from enrolling in a course before registering for an account.
  • E-commerce order management systems where order processing sequence is critical.
  • Online ticketing systems where tickets are distributed on a first-come-first-served basis.

AWS Certification Exam Practice Questions

• Questions are collected from Internet and the answers are marked as per my knowledge and understanding (which might differ with yours).
• AWS services are updated everyday and both the answers and questions might be outdated soon, so research accordingly.
• AWS exam questions are not updated to keep up the pace with AWS updates, so even if the underlying feature has changed the question might not be updated
• Open to further feedback, discussion and correction.

1. A restaurant reservation application needs the ability to maintain a waiting list. When a customer tries to reserve a table, and none are available, the customer must be put on the waiting list, and the application must notify the customer when a table becomes free. What service should the Solutions Architect recommend ensuring that the system respects the order in which the customer requests are put onto the waiting list?
   1. Amazon SNS
   2. AWS Lambda with sequential dispatch
   3. A FIFO queue in Amazon SQS
   4. A standard queue in Amazon SQS
2. A solutions architect is designing an application for a two-step order process. The first step is synchronous and must return to the user with little latency. The second step takes longer, so it will be implemented in a separate component. Orders must be processed exactly once and in the order in which they are received. How should the solutions architect integrate these components?
   1. Use Amazon SQS FIFO queues.
   2. Use an AWS Lambda function along with Amazon SQS standard queues.
   3. Create an SNS topic and subscribe an Amazon SQS FIFO queue to that topic.
   4. Create an SNS topic and subscribe an Amazon SQS Standard queue to that topic.
3. A company needs to process over 50,000 messages per second with strict ordering within each customer’s message stream. The system uses Amazon SQS FIFO queues. What should a solutions architect recommend to meet the throughput requirement?
   1. Use multiple standard queues with application-level ordering
   2. Use Amazon Kinesis Data Streams instead of SQS
   3. Enable High Throughput Mode on the FIFO queue and use unique message group IDs per customer
   4. Increase the visibility timeout to allow more concurrent processing
4. An application sends S3 event notifications to an SQS FIFO queue for ordered processing of uploaded files. The team reports that messages are not being delivered. What is the most likely cause?
   1. The SQS FIFO queue has reached its throughput limit
   2. S3 Event Notifications do not support SQS FIFO queues as a direct destination
   3. The queue’s message deduplication ID is not configured
   4. The IAM role lacks permissions to publish to the queue

References

• AWS SQS Queue Types
• AWS SQS FIFO Queues
• High Throughput for FIFO Queues
• SQS FIFO In-Flight Limit Increase (Nov 2024)
• FIFO Throughput Increase and DLQ Redrive Support