AWS S3 Object Lifecycle Management – Certification

Udemy June Discount Braincert-AWS-Certified-SA-Professional-Practice-Exam

S3 Object Lifecycle Overview

  • S3 Object lifecycle can be managed by using a lifecycle configuration, which defines how S3 manages objects during their lifetime.
  • Lifecycle configuration enables simplification of object lifecycle management, for e.g. moving of less frequently access objects, backup or archival of data for several years or permanent deletion of objects, all transitions can be controlled automatically
  • 1000 lifecycle rules can be configured per bucket
  • S3 Object Lifecycle Management rules applied to an bucket are applicable to all the existing objects in the bucket as well as the ones that will be added anew
  • S3 Object lifecycle management allows 2 types of behavior
    • Transition in which the storage class for the objects change
    • Expiration where the objects are permanently deleted
  • Lifecycle Management can be configured with Versioning, which allows storage of one current object version and zero or more non current object versions
  • Object’s lifecycle management applies to both Non Versioning and Versioning enabled buckets
  • For Non Versioned buckets
    • Transitioning period is considered from the object’s creation date
  • For Versioned buckets,
    • Transitioning period for current object is calculated for the object creation date
    • Transitioning period for non current object is calculated for the date when the object became a noncurrent versioned object
    • S3 uses the number of days since its successor was created as the number of days an object is noncurrent.
  • S3 calculates the time by adding the number of days specified in the rule to the object creation time and rounding the resulting time to the next day midnight UTC. For e.g., if an object was created at 15/1/2016 10:30 AM UTC and you specify 3 days in a transition rule, which results in 18/1/2016 10:30 AM UTC and rounded of to next day midnight time 19/1/2016 00:00 UTC.
  • Lifecycle configuration on MFA-enabled buckets is not supported.

S3 Object Lifecycle Management Rules

  1. STANDARD or REDUCED_REDUNDANCY -> (128 KB & 30 days) -> STANDARD_IA
    • Only objects with size more than 128 KB can be transitioned, as cost benefits for transitioning to STANDARD_IA can be realized only for larger objects
    • Objects must be stored for at least 30 days in the current storage class before being transitioned to the STANDARD_IA, as younger objects are accessed more frequently or deleted sooner than is suitable for STANDARD_IA
  2. STANDARD_IA -> X -> STANDARD or REDUCED_REDUNDANCY
    • Cannot transition from STANDARD_IA to STANDARD or REDUCED_REDUNDANCY
  3. STANDARD or REDUCED_REDUNDANCY or STANDARD_IA -> GLACIER
    • Any Storage class can be transitioned to GLACIER
  4. STANDARD or REDUCED_REDUNDANCY -> (1 day) -> GLACIER
    • Transitioning from Standard or RRS to Glacier can be done in a day
  5. STANDARD_IA -> (30 days) -> GLACIER
    • Transitioning from Standard IA to Glacier can be done only after 30 days or 60 days from the object creation date or non current version date
  6. GLACIER-> X -> STANDARD or REDUCED_REDUNDANCY or STANDARD_IA
    • Transition of objects to the GLACIER storage class is one-way
    • Cannot transition from GLACIER to any other storage class.
  7. GLACIER -> (90 days) -> Permanent Deletion
    • Deleting data that is archived to Glacier is free, if the objects you delete are archived for three months or longer.
    • Amazon S3 charges a prorated early deletion fee, if the object is deleted or overwritten within three months of archiving it.
  8. STANDARD or STANDARD_IA or GLACIER -> X-> REDUCED_REDUNDANCY
    • Cannot transition from any storage class to REDUCED_REDUNDANCY.
  9. Archival of objects to Amazon Glacier by using object lifecycle management is performed asynchronously and there may be a delay between the transition date in the lifecycle configuration rule and the date of the physical transition. However, AWS charges Amazon Glacier prices based on the transition date specified in the rule
  10. For a versioning-enabled bucket
    • Transition and Expiration actions apply to current versions.
    • NoncurrentVersionTransition and NoncurrentVersionExpiration actions apply to noncurrent versions and works similar to the non versioned objects except the time period is from the time the objects became noncurrent
  11. Expiration Rules
    • For Non Versioned bucket
      • Object is permanently deleted
    • For Versioned bucket
      • Expiration is applicable to the Current object only and does not impact any of the non current objects
      • S3 will insert a Delete Marker object with unique id and the previous current object becomes a non current version
      • S3 will not take any action if the Current object is a Delete Marker
      • If the bucket has a single object which is the Delete Marker (referred to as expired object delete marker), S3 removes the Delete Marker
    • For Versioned Suspended bucket
      • S3 will insert a Delete Marker object with version ID null and overwrite the any object with version ID null
  12. When an object reaches the end of its lifetime, Amazon S3 queues it for removal and removes it asynchronously. There may be a delay between the expiration date and the date at which S3 removes an object.You are not charged for storage time associated with an object that has expired.
  13. There are additional cost considerations if you put lifecycle policy to expire objects that have been in STANDARD_IA for less than 30 days, or GLACIER for less than 90 days.

AWS Certification Exam Practice Questions

  • Questions are collected from Internet and the answers are marked as per my knowledge and understanding (which might differ with yours).
  • AWS services are updated everyday and both the answers and questions might be outdated soon, so research accordingly.
  • AWS exam questions are not updated to keep up the pace with AWS updates, so even if the underlying feature has changed the question might not be updated
  • Open to further feedback, discussion and correction.
  1. Your firm has uploaded a large amount of aerial image data to S3. In the past, in your on-premises environment, you used a dedicated group of servers to oaten process this data and used Rabbit MQ, an open source messaging system, to get job information to the servers. Once processed the data would go to tape and be shipped offsite. Your manager told you to stay with the current design, and leverage AWS archival storage and messaging services to minimize cost. Which is correct?
    1. Use SQS for passing job messages, use Cloud Watch alarms to terminate EC2 worker instances when they become idle. Once data is processed, change the storage class of the S3 objects to Reduced Redundancy Storage (Need to replace On-Premises Tape functionality)
    2. Setup Auto-Scaled workers triggered by queue depth that use spot instances to process messages in SQS. Once data is processed, change the storage class of the S3 objects to Reduced Redundancy Storage (Need to replace On-Premises Tape functionality)
    3. Setup Auto-Scaled workers triggered by queue depth that use spot instances to process messages in SQS. Once data is processed, change the storage class of the S3 objects to Glacier (Glacier suitable for Tape backup)
    4. Use SNS to pass job messages use Cloud Watch alarms to terminate spot worker instances when they become idle. Once data is processed, change the storage class of the S3 object to Glacier.
  2. You have a proprietary data store on-premises that must be backed up daily by dumping the data store contents to a single compressed 50GB file and sending the file to AWS. Your SLAs state that any dump file backed up within the past 7 days can be retrieved within 2 hours. Your compliance department has stated that all data must be held indefinitely. The time required to restore the data store from a backup is approximately 1 hour. Your on-premise network connection is capable of sustaining 1gbps to AWS. Which backup methods to AWS would be most cost-effective while still meeting all of your requirements?
    1. Send the daily backup files to Glacier immediately after being generated (will not meet the RTO)
    2. Transfer the daily backup files to an EBS volume in AWS and take daily snapshots of the volume (Not cost effective)
    3. Transfer the daily backup files to S3 and use appropriate bucket lifecycle policies to send to Glacier (Store in S3 for seven days and then archive to Glacier)
    4. Host the backup files on a Storage Gateway with Gateway-Cached Volumes and take daily snapshots (Not Cost effective as local storage as well as S3 storage)

4 thoughts on “AWS S3 Object Lifecycle Management – Certification

  1. Your firm has uploaded a large amount of aerial image data to S3. In the past, in your on-premises environment, you used a dedicated group of servers to oaten process this data and used Rabbit MQ, an open source messaging system, to get job information to the servers. Once processed the data would go to tape and be shipped offsite. Your manager told you to stay with the current design, and leverage AWS archival storage and messaging services to minimize cost. Which is correct?
    Use SQS for passing job messages, use Cloud Watch alarms to terminate EC2 worker instances when they become idle. Once data is processed, change the storage class of the S3 objects to Reduced Redundancy Storage (Need to replace On-Premises Tape functionality)
    Setup Auto-Scaled workers triggered by queue depth that use spot instances to process messages in SQS. Once data is processed, change the storage class of the S3 objects to Reduced Redundancy Storage (Need to replace On-Premises Tape functionality)
    Setup Auto-Scaled workers triggered by queue depth that use spot instances to process messages in SQS. Once data is processed, change the storage class of the S3 objects to Glacier (Glacier suitable for Tape backup)
    Use SNS to pass job messages use Cloud Watch alarms to terminate spot worker instances when they become idle. Once data is processed, change the storage class of the S3 object to Glacier.

    Please explain why last option is not suitable.

    1. Key point is to stay with the current design, so using SQS for Rabbit MQ makes sense. Also the last option has missing pieces, it wouldn’t work as mentioned.

Leave a Reply

Your email address will not be published. Required fields are marked *