Preview of Azure Blob Storage Lifecycle Management

I this post, I will talk about the new automatic management of tiered Azure blob storage by a new preview Azure feature called Lifecycle Management.
 

 

Blob-Level Tiering

Azure’s blob storage allows us to move individual blobs (files) between three storage tiers in the same storage account (Hot Blob, Cool Blob, or General Purpose v2/Gpv2), optimizing the cost of our storage based on the frequency of usage:

• Hot: For frequently accessed blobs, with higher (still low cost) per-GB storage charges and lower access charges.
• Cool: For infrequently access blobs, with lower per-GB storage charges but higher access charges.
• Archive: The cheapest form of blob storage, for blobs that must be retained for legal or regulatory reasons. These blobs are very rarely otherwise accessed. Archived blobs must be moved to a cool/hot tier to be accessed and this hydration can take up to 15 hours.

By default, blob tiering is something that is not done automatically for you by Azure. You will either need to:

• Do it by hand, which probably offsets most/all savings with labor costs.
• Come up with your own solution. There is Logic App template in Azure for doing this based on each blob’s last access time.

Storage Lifecycle Management

Microsoft has launched a limited preview for a new feature called Lifecycle Management in General Purpose v2 (GPv2) storage accounts:

• West US 2 and West Central US regions to begin with.
• You must apply to become a part of the preview.

You can request access by running this AzureRM PowerShell command:

Register-AzureRmProviderFeature -FeatureName DLM -ProviderNamespace Microsoft.Storage

You can check for acceptance by running this AzureRM PowerShell command:

Get-AzureRmProviderFeature -FeatureName DLM -ProviderNamespace Microsoft.Storage

Once accepted, you can deploy a GPv2 storage account in one of the valid regions and start testing the feature. With this feature, you can create policies on a per-storage account basis to automate the movement or deletion of blobs using a set of rules. These rules are made up of a number of components.
Filters define what blobs are affected by a rule:

• blobTypes: Today, only the blockBlob is supported.
• prefixMatch: You can search for blobs starting with a certain string.

Actions specify what to do to blobs identified by a filter:

• tierToCool: Move a blob to the cool tier in the same storage account.
• tierToArchive: Move the blob to the archive tier in the same storage account.
• delete: Remove the blob from the storage account.

A Policy to Move Blobs to Cool or Archive Storage Based on Last Modification [Image Credit: Aidan Finn]

• daysAfterModificationGreaterThan: The number of days that have passed since the blob was last modified
• daysAfterCreationGreaterThan: The number of days that have passed since the blob was created in the storage account

Microsoft has shared a number of examples in its documentation of Azure blob Lifecycle Management.

Opinion

If Microsoft adds a daysAfterLastAccessGreater than action condition, then I think Lifecycle Management will offer an excellent solution for reducing the costs of Azure blob storage for those customers with huge capacities of data.