S3 Data Consistency Model

AWS S3 Data Consistency Model

~ Last updated on : ~ jayendrapatil ~ 7 Comments

AWS S3 Data Consistency Model

  • S3 Data Consistency provides strong read-after-write consistency for PUT and DELETE requests of objects in the S3 bucket in all AWS Regions
  • This behavior applies to both writes to new objects as well as PUT requests that overwrite existing objects and DELETE requests.
  • Read operations on S3 Select, S3 ACLs, S3 Object Tags, and object metadata (for example, the HEAD object) are strongly consistent.
  • Updates to a single key are atomic. for e.g., if you PUT to an existing key, a subsequent read might return the old data or the updated data, but it will never write corrupted or partial data.
  • S3 achieves high availability by replicating data across multiple servers within Amazon’s data centers. If a PUT request is successful, the data is safely stored. Any read (GET or LIST request) that is initiated following the receipt of a successful PUT response will return the data written by the PUT request.
  • S3 Data Consistency behavior examples
    • A process writes a new object to S3 and immediately lists keys within its bucket. The new object appears in the list.
    • A process replaces an existing object and immediately tries to read it. S3 returns the new data.
    • A process deletes an existing object and immediately tries to read it. S3 does not return any data because the object has been deleted.
    • A process deletes an existing object and immediately lists keys within its bucket. The object does not appear in the listing.
  • S3 does not currently support object locking for concurrent writes. S3 now supports Conditional Writes (launched Aug 2024) to handle concurrent write scenarios. for e.g. If two PUT requests are simultaneously made to the same key, the request with the latest timestamp wins. You can use conditional writes with If-None-Match or If-Match headers to prevent unintentional overwrites.
  • Updates are key-based; there is no way to make atomic updates across keys. for e.g, an update of one key cannot be dependent on the update of another key unless you design this functionality into the application.
  • S3 Object Lock is different as it allows to store objects using a write-once-read-many (WORM) model, which prevents an object from being deleted or overwritten for a fixed amount of time or indefinitely.

S3 Conditional Writes

  • S3 Conditional Writes, launched in August 2024, allow adding preconditions to write requests to coordinate concurrent writers and prevent unintentional overwrites.
  • If-None-Match header (Aug 2024)
    • Prevents overwrites of existing data by checking if an object with the same key name already exists in the bucket.
    • Expects the * (asterisk) value.
    • If an identical key name exists, the operation fails with a 412 Precondition Failed response.
    • Useful for ensuring exactly-once uploads and preventing duplicate data.
    • Supported on PutObject, CompleteMultipartUpload, and CopyObject APIs.
  • If-Match header (Nov 2024)
    • Compares the provided ETag value with the ETag of the object currently in S3.
    • If the ETag values don’t match (object was modified), the write operation fails with a 412 Precondition Failed response.
    • Enables optimistic concurrency control — check-and-set pattern without external locking.
    • Helps coordinate simultaneous writes and prevents multiple concurrent writers from unintentionally overwriting objects.
    • Supported on PutObject, CompleteMultipartUpload, and CopyObject APIs.
  • Conditional Write Enforcement via Bucket Policy (Nov 2024)
    • Bucket owners can use s3:if-none-match or s3:if-match condition keys in bucket policies to mandate the use of conditional headers.
    • Requests without the required conditional headers will be denied with a 403 Access Denied error.
  • Conditional Writes for Copy Operations (Oct 2025)
    • Extended conditional write support to CopyObject operations.
    • Allows verifying if an object exists or has been modified in the destination bucket before copying.
    • Uses s3:if-match and s3:if-none-match condition keys enforceable via bucket policies.
  • Error Responses
    • 412 Precondition Failed — Precondition not met (object exists or ETag mismatch).
    • 409 Conflict — A concurrent delete request succeeded before the conditional write completed.
    • 404 Not Found — Object no longer exists when using If-Match (concurrent delete).
  • There is no additional charge for conditional writes — standard request pricing applies.
  • Use cases include leader election, distributed locking, write-once data pipelines, and multi-writer applications without external coordination systems like DynamoDB.

S3 Default Data Integrity Protections

  • Starting December 2024, S3 provides default data integrity protections for new object uploads.
  • The latest AWS SDKs automatically calculate CRC-based checksums for uploads as data is transmitted over the network.
  • S3 independently verifies these checksums and accepts objects after confirming data integrity was maintained in transit.
  • If no checksum is provided on upload, S3 automatically calculates and applies a CRC64NVME checksum as default integrity protection.
  • Checksums can be requested during download to verify data consistency.
  • This ensures end-to-end data integrity without requiring application-level checksum implementation.

S3 Consistency with S3 Express One Zone

  • S3 Express One Zone (launched Nov 2023) provides strong read-after-write consistency with single-digit millisecond data access.
  • Objects are stored in directory buckets in a single Availability Zone, co-located with compute resources for lowest latency.
  • Delivers up to 10x faster performance and up to 50% lower request costs compared to S3 Standard.
  • Supports conditional deletes (Nov 2024) — can evaluate whether an object is unchanged before deleting it, improving data durability in high-concurrency scenarios.
  • Provides the same strong consistency model as S3 Standard for all operations.

S3 Legacy Consistency Model (Historical Reference)

  • S3 provides strong Read-after-Write consistency for PUTS of new objects
    • For a PUT request, S3 synchronously stores data across multiple facilities before returning SUCCESS
    • A process writes a new object to S3 and will be immediately able to read the Object i.e. PUT 200 -> GET 200
    • A process writes a new object to S3 and immediately lists keys within its bucket. Until the change is fully propagated, the object might not appear in the list.
    • However, if a HEAD or GET request to a key name is made before the object is created, then create the object shortly after that, a subsequent GET might not return the object due to eventual consistency. i.e. GET 404 -> PUT 200 -> GET 404
  • S3 provides Eventual Consistency for overwrite PUTS and DELETES in all regions.
    • For updates and deletes to Objects, the changes are eventually reflected and not available immediately i.e. PUT 200 -> PUT 200 -> GET 200 (might be older version) OR DELETE 200 -> GET 200
    • if a process replaces an existing object and immediately attempts to read it, S3 might return the prior data till the change is fully propagated
    • if a process deletes an existing object and immediately attempts to read it, S3 might return the deleted data until the deletion is fully propagated
    • if a process deletes an existing object and immediately lists keys within its bucket. Until the deletion is fully propagated, S3 might list the deleted object.
  • Note: Since December 2020, S3 provides strong read-after-write consistency for ALL operations (GET, PUT, LIST, DELETE, tags, ACLs, metadata). The eventual consistency model above is no longer applicable but retained for historical and exam reference.

AWS Certification Exam Practice Questions

  • Questions are collected from Internet and the answers are marked as per my knowledge and understanding (which might differ with yours).
  • AWS services are updated everyday and both the answers and questions might be outdated soon, so research accordingly.
  • AWS exam questions are not updated to keep up the pace with AWS updates, so even if the underlying feature has changed the question might not be updated
  • Open to further feedback, discussion and correction.
  1. Which of the following are valid statements about Amazon S3? Choose 2 answers
    1. S3 provides read-after-write consistency for any type of PUT or DELETE. (S3 now provides strong read-after-write consistency)
    2. Consistency is not guaranteed for any type of PUT or DELETE.
    3. A successful response to a PUT request only occurs when a complete object is saved
    4. Partially saved objects are immediately readable with a GET after an overwrite PUT.
    5. S3 provides eventual consistency for overwrite PUTS and DELETES
  2. A customer is leveraging Amazon Simple Storage Service in eu-west-1 to store static content for web-based property. The customer is storing objects using the Standard Storage class. Where are the customers’ objects replicated?
    1. Single facility in eu-west-1 and a single facility in eu-central-1
    2. Single facility in eu-west-1 and a single facility in us-east-1
    3. Multiple facilities in eu-west-1
    4. A single facility in eu-west-1
  3. A user has an S3 object in the US Standard region with the content “color=red”. The user updates the object with the content as “color=”white”. If the user tries to read the value 1 minute after it was uploaded, what will S3 return?
    1. It will return “color=white” (strong read-after-write consistency — S3 provides strong consistency for all operations since December 2020)
    2. It will return “color=red”
    3. It will return an error saying that the object was not found
    4. It may return either “color=red” or “color=white” i.e. any of the value (Eventual Consistency — No longer applicable)
  4. A company has multiple applications writing to the same S3 object simultaneously. They want to ensure that an application only writes to the object if no other application has modified it since it was last read. What S3 feature should they use?
    1. S3 Object Lock with Governance mode
    2. S3 Versioning with MFA Delete
    3. S3 Conditional Writes with the If-Match header (If-Match compares ETag to ensure object hasn’t changed since last read — optimistic concurrency control)
    4. S3 Cross-Region Replication
  5. An application needs to ensure that an object is uploaded to S3 only if no object with the same key already exists. Which approach provides this guarantee at the storage layer without external coordination?
    1. Check with a GET request before uploading with PUT
    2. Use S3 Versioning to track duplicates
    3. Use DynamoDB as a distributed lock manager
    4. Use the If-None-Match header with PutObject (If-None-Match with * value ensures the PUT only succeeds if no object with that key exists — atomic check-and-put)
  6. Which of the following statements about S3 Conditional Writes are correct? (Choose 2)
    1. The If-Match header can be used to update an object only if its ETag matches the expected value
    2. Conditional writes require S3 Object Lock to be enabled on the bucket
    3. Bucket policies can enforce conditional writes using s3:if-none-match condition keys
    4. Conditional writes are only supported on S3 Express One Zone directory buckets
    5. Conditional writes incur additional per-request charges

References