S3 Object Lifecycle Management

• S3 Object lifecycle can be managed by using a lifecycle configuration, which defines how S3 manages objects during their lifetime.
• Lifecycle configuration enables simplification of object lifecycle management, for e.g. moving of less frequently access objects, backup or archival of data for several years, or permanent deletion of objects.
• S3 controls all transitions automatically.
• Lifecycle Management rules applied to a bucket are applicable to all the existing objects in the bucket as well as the ones that will be added anew.
• S3 Object lifecycle management allows the following types of actions:
  • Transition – changes the storage class for the objects.
  • Expiration – permanently deletes objects.
  • AbortIncompleteMultipartUpload – aborts incomplete multipart uploads after a specified number of days since initiation.
• Lifecycle Management can be configured with Versioning, which allows storage of one current object version and zero or more non-current object versions.
• Object’s lifecycle management applies to both Non Versioning and Versioning enabled buckets.
• For Non Versioned buckets:
  • Transitioning period is considered from the object’s creation date.
• For Versioned buckets:
  • Transitioning period for the current object is calculated from the object creation date.
  • Transitioning period for a non-current object is calculated from the date when the object became a noncurrent versioned object.
  • S3 uses the number of days since its successor was created as the number of days an object is noncurrent.
• S3 calculates the time by adding the number of days specified in the rule to the object creation time and rounding the resulting time to the next day midnight UTC for e.g. if an object was created at 15/1/2016 10:30 AM UTC and you specify 3 days in a transition rule, which results in 18/1/2016 10:30 AM UTC and rounded to next day midnight time 19/1/2016 00:00 UTC.
• Lifecycle configuration on MFA-enabled buckets is not supported.
• 1000 lifecycle rules can be configured per bucket (not adjustable).
• General purpose buckets – You can’t use a bucket policy to prevent deletions or transitions by an S3 Lifecycle rule. Even if your bucket policy denies all actions for all principals, your S3 Lifecycle configuration still functions as normal.
• Directory buckets (S3 Express One Zone) – S3 Lifecycle supports expiration actions only (no transitions between storage classes). Bucket policies can block lifecycle deletions in directory buckets, requiring proper CreateSession permissions.

S3 Lifecycle Filters

• Lifecycle rules can filter objects using the following criteria:
  • Prefix – applies rule to objects with a specific key prefix.
  • Object Tags – applies rule to objects with one or more specific tags.
  • Object Size – filters by ObjectSizeGreaterThan and/or ObjectSizeLessThan to specify minimum and maximum object sizes.
  • And operator – combines multiple filter criteria (prefix, tags, and size).
• If the Filter element is left empty, the Lifecycle Rule applies to all objects in the bucket.
• Object size filters allow overriding the default 128 KB minimum transition size restriction.

S3 Object Lifecycle Management Rules

Supported Lifecycle Transitions (Waterfall Model)

Amazon S3 supports a waterfall model for transitioning between storage classes:

• S3 Standard → S3 Standard-IA, S3 Intelligent-Tiering, S3 One Zone-IA, S3 Glacier Instant Retrieval, S3 Glacier Flexible Retrieval, or S3 Glacier Deep Archive.
• S3 Standard-IA → S3 Intelligent-Tiering, S3 One Zone-IA, S3 Glacier Instant Retrieval, S3 Glacier Flexible Retrieval, or S3 Glacier Deep Archive.
• S3 Intelligent-Tiering → depends on the current access tier:
  • Frequent Access or Infrequent Access tier → S3 One Zone-IA, S3 Glacier Instant Retrieval, S3 Glacier Flexible Retrieval, or S3 Glacier Deep Archive.
  • Archive Instant Access tier → S3 Glacier Instant Retrieval, S3 Glacier Flexible Retrieval, or S3 Glacier Deep Archive.
  • Archive Access tier → S3 Glacier Flexible Retrieval or S3 Glacier Deep Archive.
  • Deep Archive Access tier → S3 Glacier Deep Archive.
• S3 One Zone-IA → S3 Glacier Flexible Retrieval or S3 Glacier Deep Archive.
• S3 Glacier Instant Retrieval → S3 Glacier Flexible Retrieval or S3 Glacier Deep Archive.
• S3 Glacier Flexible Retrieval → S3 Glacier Deep Archive.

Lifecycle Transitions Constraints

1. Object Size Default Behavior (Updated September 2024)
   • Objects smaller than 128 KB will not transition to any storage class by default.
   • This applies to all storage classes (updated from previous behavior that allowed small objects to transition to Glacier classes).
   • To allow smaller objects to transition, add an object size filter (ObjectSizeGreaterThan or ObjectSizeLessThan) to your rule.
   • Configurations created before September 2024 retain the previous behavior unless modified.
2. Minimum 30 Days for S3 Standard-IA or S3 One Zone-IA
   • Objects must be stored for at least 30 days in the current storage class before being transitioned to S3 Standard-IA or S3 One Zone-IA.
   • Similarly, noncurrent objects in versioned buckets must be at least 30 days noncurrent.
3. Minimum Storage Duration Charges
   • S3 Standard-IA / S3 One Zone-IA – 30 days minimum.
   • S3 Glacier Instant Retrieval – 90 days minimum.
   • S3 Glacier Flexible Retrieval – 90 days minimum.
   • S3 Glacier Deep Archive – 180 days minimum.
   • Deleting or transitioning objects before the minimum duration incurs prorated early deletion fees.
   • You can’t create a single lifecycle rule that transitions through storage classes faster than minimum durations allow (e.g., transition to Glacier Instant Retrieval after 4 days, then to Deep Archive after 20 days). Two separate rules are needed, but minimum duration charges still apply.
4. Replication and Lifecycle Interaction (Updated March 2026)
   • S3 Lifecycle now prevents expiration and transition actions on objects with PENDING or FAILED replication status.
   • This ensures lifecycle does not act on objects until they have successfully replicated to their destination bucket.
   • For versioning-enabled or versioning-suspended buckets, objects with a Pending or Failed replication status cannot be transitioned.
5. Archival Storage Overhead
   • For each object archived to S3 Glacier Flexible Retrieval or S3 Glacier Deep Archive, S3 adds 8 KB metadata (charged at S3 Standard rates) + 32 KB index data (charged at Glacier rates) = 40 KB total overhead per object.
   • Consider aggregating small objects into larger ones to reduce overhead costs.
6. Expiration Rules
   • For Non Versioned bucket:
     • Object is permanently deleted.
   • For Versioned bucket:
     • Expiration applies to the Current object only and does not impact noncurrent versions.
     • S3 inserts a Delete Marker with a unique version ID; the previous current object becomes noncurrent.
     • S3 will not take any action if the Current object is already a Delete Marker.
     • If the bucket has a single object which is the Delete Marker (expired object delete marker), S3 removes the Delete Marker.
   • For Versioned Suspended bucket:
     • S3 inserts a Delete Marker with version ID null and overwrites any object with version ID null.
7. NoncurrentVersionExpiration
   • Allows permanent deletion of noncurrent object versions after a specified number of days.
   • NewerNoncurrentVersions – Specifies how many noncurrent versions to retain (up to 100). S3 permanently deletes any additional noncurrent versions beyond this number.
   • Helps control storage costs in versioning-enabled buckets by keeping only a defined number of older versions.
8. Asynchronous Processing
   • When an object reaches the end of its lifetime, S3 queues it for removal and removes it asynchronously.
   • There may be a delay between the expiration/transition date and the actual action.
   • However, billing changes occur based on the rule satisfaction date (except for transitions to S3 Intelligent-Tiering where billing changes after physical transition).
   • S3 Lifecycle rules run once each day.
9. Tag-based Filter Evaluation
   • S3 evaluates objects against tag-based filters daily.
   • At execution time, S3 re-evaluates the object’s current tags. If the triggering tag is no longer present, the transition does not proceed.
   • Removing a tag does not guarantee immediate cancellation – the action may execute before the tag removal is observed.

S3 Lifecycle Event Notifications

• S3 can send event notifications when lifecycle actions are performed:
  • s3:LifecycleTransition – notifies when an object is transitioned from one storage class to another.
  • s3:LifecycleExpiration:Delete – notifies when an object is permanently deleted (unversioned bucket) or when an object version is permanently deleted.
  • s3:LifecycleExpiration:DeleteMarkerCreated – notifies when S3 Lifecycle creates a delete marker for a current version in a versioned bucket.
• Notifications can be published to SNS, SQS, or Lambda.
• Useful for tracking and auditing lifecycle actions.

S3 Intelligent-Tiering and Lifecycle

• S3 Intelligent-Tiering automatically moves objects through access tiers based on access patterns:
  • Frequent Access – default tier for newly uploaded objects.
  • Infrequent Access – objects not accessed for 30 consecutive days (40% lower cost).
  • Archive Instant Access – objects not accessed for 90 consecutive days (68% lower cost than Infrequent Access).
  • Archive Access (optional) – objects not accessed for 90-730 days (configurable).
  • Deep Archive Access (optional) – objects not accessed for 180-730 days (configurable).
• Objects accessed in any tier are automatically moved back to the Frequent Access tier.
• Lifecycle rules can transition objects from any storage class to S3 Intelligent-Tiering.
• No retrieval charges when objects move between tiers within Intelligent-Tiering.
• A monthly monitoring and automation charge per object applies.
• No minimum object size requirement (no 128 KB minimum) for Intelligent-Tiering.

S3 Express One Zone Lifecycle (Directory Buckets)

• S3 Express One Zone (directory buckets) supports S3 Lifecycle expiration rules since November 2024.
• Supported actions: Expiration (delete objects) only. Transition actions between storage classes are not supported.
• Lifecycle rules can filter by prefix or object size.
• Unlike general purpose buckets, bucket policies in directory buckets can block lifecycle deletions – requires proper CreateSession permissions.
• Useful for automatically cleaning up temporary or short-lived data in high-performance workloads.

AWS Certification Exam Practice Questions

• Questions are collected from Internet and the answers are marked as per my knowledge and understanding (which might differ with yours).
• AWS services are updated everyday and both the answers and questions might be outdated soon, so research accordingly.
• AWS exam questions are not updated to keep up the pace with AWS updates, so even if the underlying feature has changed the question might not be updated
• Open to further feedback, discussion and correction.

1. If an object is stored in the Standard S3 storage class and you want to move it to Glacier, what must you do in order to properly migrate it?
   1. Change the storage class directly on the object.
   2. Delete the object and re-upload it, selecting Glacier as the storage class.
   3. None of the above.
   4. Create a lifecycle policy that will migrate it after a minimum of 30 days. (Any object uploaded to S3 must first be placed into either the Standard, Reduced Redundancy, or Infrequent Access storage class. Once in S3 the only way to move the object to glacier is through a lifecycle policy. NOTE: As of current AWS updates, you can also use the CopyObject API or S3 console to directly change storage class, but lifecycle policies remain the automated approach.)
2. A company wants to store their documents in AWS. Initially, these documents will be used frequently, and after a duration of 6 months, they would not be needed anymore. How would you architect this requirement?
   1. Store the files in Amazon EBS and create a Lifecycle Policy to remove the files after 6 months.
   2. Store the files in Amazon S3 and create a Lifecycle Policy to remove the files after 6 months.
   3. Store the files in Amazon Glacier and create a Lifecycle Policy to remove the files after 6 months.
   4. Store the files in Amazon EFS and create a Lifecycle Policy to remove the files after 6 months.
3. Your firm has uploaded a large amount of aerial image data to S3. In the past, in your on-premises environment, you used a dedicated group of servers to process this data and used Rabbit MQ, an open source messaging system, to get job information to the servers. Once processed the data would go to tape and be shipped offsite. Your manager told you to stay with the current design, and leverage AWS archival storage and messaging services to minimize cost. Which is correct?
   1. Use SQS for passing job messages, use CloudWatch alarms to terminate EC2 worker instances when they become idle. Once data is processed, change the storage class of the S3 objects to Reduced Redundancy Storage (Need to replace On-Premises Tape functionality. Also, RRS is no longer recommended.)
   2. Setup Auto-Scaled workers triggered by queue depth that use spot instances to process messages in SQS. Once data is processed, change the storage class of the S3 objects to Reduced Redundancy Storage (Need to replace On-Premises Tape functionality. Also, RRS is no longer recommended.)
   3. Setup Auto-Scaled workers triggered by queue depth that use spot instances to process messages in SQS. Once data is processed, change the storage class of the S3 objects to Glacier (Glacier suitable for Tape backup. Note: Current terminology is S3 Glacier Flexible Retrieval.)
   4. Use SNS to pass job messages use CloudWatch alarms to terminate spot worker instances when they become idle. Once data is processed, change the storage class of the S3 object to Glacier.
4. You have a proprietary data store on-premises that must be backed up daily by dumping the data store contents to a single compressed 50GB file and sending the file to AWS. Your SLAs state that any dump file backed up within the past 7 days can be retrieved within 2 hours. Your compliance department has stated that all data must be held indefinitely. The time required to restore the data store from a backup is approximately 1 hour. Your on-premise network connection is capable of sustaining 1gbps to AWS. Which backup methods to AWS would be most cost-effective while still meeting all of your requirements?
   1. Send the daily backup files to Glacier immediately after being generated (will not meet the RTO)
   2. Transfer the daily backup files to an EBS volume in AWS and take daily snapshots of the volume (Not cost effective)
   3. Transfer the daily backup files to S3 and use appropriate bucket lifecycle policies to send to Glacier (Store in S3 for seven days and then archive to S3 Glacier Flexible Retrieval)
   4. Host the backup files on a Storage Gateway with Gateway-Cached Volumes and take daily snapshots (Not Cost-effective as local storage as well as S3 storage)
5. A company has millions of small objects (5 KB each) stored in S3 Standard. They want to create a lifecycle rule to transition these objects to S3 Glacier Flexible Retrieval after 90 days. What should they consider?
   1. Objects smaller than 128 KB will not be transitioned by default. They should use an object size filter to explicitly allow small object transitions, and also consider the 40 KB per-object storage overhead in Glacier.
   2. Objects smaller than 128 KB are automatically transitioned to Glacier with no restrictions.
   3. Small objects cannot be stored in any Glacier storage class.
   4. They must first transition to Standard-IA before transitioning to Glacier.
6. A company uses S3 Cross-Region Replication (CRR) and S3 Lifecycle rules on the same bucket. Some objects have a replication status of FAILED. What happens when the lifecycle expiration rule is triggered for these objects?
   1. The objects are immediately deleted as per the lifecycle rule.
   2. S3 Lifecycle pauses expiration and transition actions on objects with PENDING or FAILED replication status until replication succeeds or is resolved.
   3. The objects are transitioned to Glacier despite the failed replication.
   4. The lifecycle rule is permanently disabled for the bucket.
7. Which of the following is a valid S3 Lifecycle transition path?
   1. S3 Glacier Deep Archive → S3 Standard
   2. S3 One Zone-IA → S3 Standard-IA
   3. S3 Glacier Instant Retrieval → S3 Glacier Flexible Retrieval
   4. S3 Glacier Flexible Retrieval → S3 Glacier Instant Retrieval
8. A company wants to retain only the 5 most recent noncurrent versions of objects in a versioned S3 bucket and delete all older versions. Which lifecycle configuration should they use?
   1. Set an Expiration action with Days = 5
   2. Use a transition rule to move old versions to Glacier
   3. Configure NoncurrentVersionExpiration with NewerNoncurrentVersions set to 5
   4. Enable S3 Object Lock with a retention period of 5 days

References

• AWS S3 Object Lifecycle Management
• Transitioning objects using Amazon S3 Lifecycle
• Adding filters to Lifecycle rules
• Configuring S3 Lifecycle event notifications
• Managing storage costs with Amazon S3 Intelligent-Tiering
• Working with S3 Lifecycle for directory buckets