Gaurav Mantri's Personal Blog.

Understanding Azure Storage Blob Access Tiers

It has been really-really long time that I have written a blog post. Past year or so has been simply crazy. From acquiring Cerebrata back to building a brand new product (Cerulean) from scratch, things kept me quite busy.

In this blog post we will talk about blob access tiers. We will talk about what these are and some guidance around when to use them. We will talk about cost implications of storing blobs in each access tier. Lastly we will talk about a tool that I built for estimating the costs of storing blobs in each access tier.

Blob Access Tiers

As we all know Azure Storage provides a reliable and highly available object storage to store massive amounts of data for a very cheap price. Blob access tiers is a functionality provided by Azure Storage to store your blobs in different access tiers based on how these blobs are accessed (hence the term “Blob Access Tier” Smile).

Currently there are three access tiers in which you can store the blobs:

  • Hot
  • Cool
  • Archive (Preview)

Now let’s talk about these tiers in somewhat more details.

“Hot” Access Tier

Typically you would want to put your blobs in “Hot” access tier when these blobs are accessed very frequently. For example, if you have a website and you’re storing the images, style sheets and JavaScript files required by that website in Azure Storage, you would want to put them in this tier as they are accessed very frequently.

For blobs stored in “Hot” access tier, you pay a higher storage price (compared to storage price in cooler tiers) but you pay lower transaction price (again compared to transaction price in cooler tier) when these blobs are accessed thus making this tier optimal for frequently accessed blobs.

“Cool” Access Tier

Typically you would want to put your blobs in “Cool” access tier when these blobs are accessed not that frequently yet you want them to be instantly available when needed. Good candidates for putting blobs in this access tier are the blobs holding recent backups (that you may require instantly to recover from), any content that is not accessed recently but when accessed should be available instantly.

For blobs stored in “Cool” access tier, you pay a relatively lower storage price (compared to storage prices for hot tier) but you pay higher transaction price (again compared to transaction prices for hot tier) when these blobs are accessed thus making this tier optimal for rarely accessed yet instantly available blobs.

“Archive” Access Tier

This is the latest access tier announced by Azure Storage and is currently in preview (at the time of writing of this blog).

Typically you would want to put your blobs in “Archive” access tier when these blobs are accessed not that frequently (or not at all) and you’re fine with latency of a few hours in accessing these blobs when needed. Good candidates for putting blobs in this tier are old backups, email archives and other things you would want to keep for compliance reasons.

For blobs stored in “Archive” access tier, you pay a lot less for storage price (compared to storage prices for hot tier) but you pay significantly higher transaction price (again compared to transaction prices for hot tier) when these blobs are accessed thus making this tier optimal for rarely accessed blobs.

A few more things about this access tier:

  • This access tier is currently in preview only thus its availability is limited.
  • During preview phase it is only available in US East 2 region and is only available for a Local Redundant Storage (LRS) Blob Storage accounts. Availability for other regions and other storage accounts will be announced later.
  • During preview phase you will need to enable your Azure Subscription to use this feature before you can use it. Please see this blog post from Azure Storage team for instructions on how to enable this feature in your Azure Subscription using either PowerShell or CLI tools: Announcing the public preview of Azure Archive Blob Storage and Blob-Level Tiering.
  • During preview phase the storage pricing for this tier is about 10% of that of hot tier however the pricing may go up when this feature becomes generally available. So a 90% price reduction in storage cost should not be a sole criteria for storing blobs in this tier.
  • During preview phase it is not possible for you to directly upload a file in this access tier. You must first upload a blob in hot access tier and then perform “Set Blob Tier” operation on that blob to move the blob to archive tier. To do it via .Net SDK, please use SDK version 8.4 or greater.
  • During preview phase it is not possible for you to directly read (for downloading purpose) a blob in this access tier. You must first convert that blob’s tier to either hot or cool (a process known as rehydration that could take hours) and then read the blob.
  • In fact, during preview phase it is not possible for you to perform any operations on blobs in this access tier other than listing the blobs and fetching properties of such blobs. If you really need to perform other operations (like setting metadata), you must first rehydrate the blob, perform operation and then change blob tier to archive again.

Blob Access Tiers & Cost Implications

Typically when people look at cloud object storage (like Azure Storage), they often consider only the cost of storing the objects there. They forget other costs associated with storing and accessing objects.

When using Azure Storage, typically these are the costs you can incur:

  • Storage Costs: These are the costs of storing objects in Azure Storage. in Azure you pay for the amount of time objects are stored in Azure.
  • Read Transactions Costs: Every time a blob or its properties are read, you incur these charges.
  • Write Transaction Costs: Every time a blob is written or updated, you incur these charges.
  • Data Egress Costs: Whenever blob data is transferred outside of the Azure region it is hosted in, you incur these charges.
  • Data Retrieval Costs: Whenever a blob is read from a cool or archive access tier, you incur these charges.
  • Data Write Costs: Whenever a blob is written to a cool or archive access tier, you incur these charges.

When considering the storage costs, one must consider all of these costs.

Which Access Tier Is Right For Me?

If I were to make a decision tree for this, the very first question that I will ask myself regarding this would be “Is the data stored in blobs accessed frequently?”.

If the answer to this question is yes, then it is a no-brainer that “Hot” access tier is right for you. You may pay higher storage costs but reduced transaction costs will offset this cost.

If the answer to this question is no, then the next question I will ask myself is “Whenever I need this data, do I need this data to be readily available or am I ok to wait for a few hours to get this data?

If the answer to this question is “I need this data to be readily available”, then “Cool” access tier is right for you.

However if your answer is “I am ok to wait for a few hours to get this data”, then “Archive” access tier may be right for you. Please note the emphasis I have put on “may” above. The reason for this is that a blob from an archive access tier will need to go through a rehydration process which essentially converts a blob’s access tier from “Archive” to “Hot”. Not only it is time consuming (but you’re ok with that Smile) but also an expensive operation. Also not to mention that once you’re done reading the blob, you will need to put this back in “Archive” tier from “Hot” tier and you will pay for those transactions as well.

One big question that still remains is how do you determine if the data stored in blobs accessed frequently or not? There are a few ways by which you can determine this:

  • Check blob’s last modified date: Though not an accurate measure but you can use a blob’s last modified date to determine if a blob is that frequently accessed or not. This will work for things like backups and log files that you store in Azure Storage. Typically once they are written, they are rarely accessed. However it may fail in scenarios where you’re storing content like website logos, images etc. Even though they are also rarely modified but are still frequently accessed. So you will have to be careful when choosing this criteria.
  • Check storage analytics logs: If you have enabled analytics logs for your storage account, every time a blob is accessed an entry is made in storage analytics logs. By checking for blobs in these logs can give you an accurate idea about whether or not a blob is accessed in say last 30/60 days or not. However this is a very cumbersome process as these log files are delimited text files and you would need to first download these log files, parse them to find the occurrences of blobs of your interest.

Blob Tier Analysis Tool

In this year’s Microsoft Ignite conference I had the privilege of speaking at the event. One of the sessions I spoke at was about this archive tier and for that I had prepared a tool called “Blob Tier Analysis Tool”.

This is a command line tool that can analyze the contents of your storage account and will tell you about the potential cost savings if you were to move these blobs into a different tier (say from Hot/Cool to Archive). It will also change the blob’s tier for you as well. Furthermore it can also analyze the contents of a local folder/file share on your computer and tell you about the potential costs if you were to store these files in Azure Storage.

The source code for this tool is available on Github so please download and give it a try: https://github.com/Azure-Samples/storage-dotnet-blob-tier-analysis-tool.

Summary

That’s it for this post. I hope you have found this post useful. If you find any issues with this post, please let me know and I will fix them ASAP.

Until next time!!!

My Thoughts On Current Euphoria Over Startups In India

Nowadays everyone wants to do a startup. There is a general feeling that if they are not doing a startup, they are not doing anything . I was listening to an old song by Amitabh Bachchan and took the liberty of modifying it to reflect the current … [Continue reading]

Azure Storage – Shared Access Signature Enhancements

Over the past few months, Azure Storage Team released two major upgrades. Both of these upgrades involve some really interesting new features and improvements. Among these new features and improvements include changes to Shared Access Signature (SAS) … [Continue reading]

Azure Icon Font For Your Web Application

For both Cloud Portam website and application, when it comes to icons we don’t use images. Almost invariably, we use vector icons and icon fonts (like Font Awesome which BTW are very aptly named; they are indeed really-really awesome :)). Where … [Continue reading]

What’s New In Azure Storage

It’s been a while that I wrote a blog post about Azure Storage :). Earlier this month, Azure Storage Team released a new version of Storage Service and included a lot of awesome goodness! In this blog post, I will try to summarize those. So … [Continue reading]

Azure Service Bus – As I Understand It: Part II (Queues & Messages)

Continuing from my previous post about Azure Service Bus, in this post I will share my learning about Queues & Messages. The focus of this post will be about some of the undocumented things I found as we implemented support for Queues and … [Continue reading]

Azure Service Bus – As I Understand It: Part I (Overview)

Recently we started working on including support for Azure Service Bus in Cloud Portam. Prior to this, I had no experience with this service though it has been around for quite some time and I always wanted to try this out but one thing or another … [Continue reading]

Understanding Azure Premium Storage – A Developer Perspective

As you may already know Azure Premium Storage is the latest offering from Azure Storage which is designed specifically to handle Virtual Machine workloads. A few weeks back we included support for managing Premium Storage in Cloud Portam. There were … [Continue reading]

Prarambh 2015 – A Recap

Over the weekend of Feb 20 - 22nd 2015, Indian Institute of Management (IIM) Udaipur’s Entrepreneurship Cell “Saksham” organized a 32 hour startup challenge event (Prarambh). This was the 2nd time an event like this was organized (and I'm extremely … [Continue reading]

What Startup Founders Can Learn From Ocean’s Eleven Movie

I’m a huge fan of movie Ocean’s Eleven. In fact, I watch this movie every time it comes on TV. For those who don’t know about this movie, in short it is a heist film where George Clooney along with 10 other folks rob three Las Vegas casinos in one … [Continue reading]