S3 Lifecycle Management Policies Overview

Different storage tiers have different cost. S3 Lifecycle management provides the ability to define the lifecycle of your object to transition to different storage tiers based on different factors.

For example, you can set infrequently accessed objects to move into lower cost storage tier (like S3-IA) after a period of time. After another period, it can be moved into Amazon Glacier for archive and compliance, and eventually deleted.

As another example, the lifecycle policy that expires incomplete multipart uploads allows you to save on costs by limiting the time non-completed multipart uploads are stored.

What can you do with lifecycle transition policy?

You can set lifecycle transition policy to:

1. automatically migrate S3 objects to S3-IA and/or Glacier, based on the age of the data.

2. set lifecycle expiration policies to automatically remove objects based on the age of the object.

3. set a policy for multipart upload expiration, which expires incomplete multipart upload based on the age of the upload.

How do you set lifecycle transition policy?

1. You can set up and manage lifecycle policies in the Management Console, S3 REST API, AWS SDKs, or AWS CLI.

2. You can specify the policy at the prefix or at the bucket level.

3. In each policy rule you can specify a prefix, a time period, a transition to Standard - IA or Amazon Glacier, and/or an expiration.

4. To apply the rule to an individual object, specify the key name instead of prefix. To apply the rule to a set of objects, specify their common prefix (e.g. “logs/”). For expiration period, provide the number of days from creation date (i.e. age) after which you want your objects removed.

5. After an Object Expiration rule is added, the rule is applied to objects that already exist in the bucket as well as new objects added to the bucket. Once objects are past their expiration date, they are identified and queued for removal.

6. You can set Expiration rules for your versioning-enabled or versioning-suspended buckets as well.

Important Notes on Lifecycle Management (Exam Tips)

1. Standard-IA has a minimum retention period of 30 days and 128KB minimum objectsize. A minimum of 30 days is required before transitioning to the Standard-IA storage class and file size should be minimum 128KB.

2. An object must remain in the Standard-IA storage class for a minimum of 30 days before transitioning to the Glacier storage class.

3. If you do not have a transition to Standard-IA, there is no restriction on minimum days for directly transitioning to the Glacier storage class.

4. S3 objects that are stored in Glacier through lifecycle rules are only accessible through the S3 APIs or the S3 Management Console. This is because S3 maintains the mapping between your object name and Glacier’s system-defined identifier. First you need to initiate the retrieval request and once the job is complete it is accessible through an Amazon S3 GET object request.

Important Notes on Lifecycle Management Pricing (Exam Tips)

1. There is no additional cost to set up and apply lifecycle policies. A transition request is charged per object when an object becomes eligible for transition according to the lifecycle rule.

2. You will not be billed for storage for objects on or after their expiration date, though you may still be able to access those objects while they are in queue before they are removed.

3. As with standard delete requests, S3 doesn’t charge you for removing objects using Object Expiration.