Last week we saw a lot of new features introduced in Windows Azure Platform. We also saw that all of these new features can be accessed either via new and improved Windows Azure Portal or using Service Management REST API (http://msdn.microsoft.com/en-us/library/windowsazure/ee460799). In fact, Windows Azure Portal is built on this API itself. What this means is that if you’re not happy with how things work in Windows Azure Portal and would like to do things differently (or automate certain processes), you could roll out your own solution based on this API.
I was eagerly waiting for changes to Storage Service and yesterday they were announced at Tech Ed 2012. In this post, I will summarize those changes. When compared to changes to Service Management API, even though the changes are rather smaller in number but they are quite significant changes. From working with Windows Azure for quite some time now, I know that these features have been asked by Windows Azure users for sometime and the impact these changes will make. Kudos to Windows Azure Storage team to listen to their users and implementing these enhancements to make Windows Azure Storage even more attractive to use (after 10x price reduction and geo-replication).
The new version of storage service to take advantage of these new features is 2012-02-12. If you’re writing your custom code to implement Storage REST API, please ensure to pass this value for “x-ms-version” request header to take advantage of these new features.
So here goes the list of new features!!!
Copying Blobs across Storage Accounts (A Very BIG Yay!!!)
In the previous versions, you could only copy blob from one blob container to another one in same storage account. In the new version, you could copy blobs from one storage account to another storage account. This is a very big improvement. A use case scenario for this would be taking backup of blob storage. Currently you would need to fetch the contents of a blob from “source” storage account and then push the contents to “target” storage account. NOT ANY MORE!!! Now the platform takes care of this for you. More PaaS for you!!!
Some comments on this functionality:
- You can only copy blob from one storage account (source) to another storage account (target), only if the target storage account is created after 7th June 2012.
- Now the copy blob operation is an asynchronous operation. In the earlier versions, it was a synchronous operation.
- To perform copy operation at REST API level, you would need to specify complete URL of the source blob in “http[s]://<storage account endpoint>.<blob container name>.<blob name>[?snapshot=<snapshot value>]” format. Earlier you just specified the source blob in “<blob container name>/<blob name>[?snapshot=<snapshot value]” format.
- When copying blobs across storage accounts, the source blob must either be publicly accessible i.e. must reside in a blob container with public ACL or if the source blob is in a blob container with private ACL, then the URL for source blob must be a signed URL generated using a Shared Access Signature which allows “Read” access to the source blob.
- (THIS IS HUGE) For copying, it is NOT necessary to have your source blob in Windows Azure Storage. It could be outside of Windows Azure Storage (e.g. objects in Amazon S3) provided either the source blob is publicly accessible or accessible via some form of signed URL. This makes it real easy for you to copy your blob contents from one cloud provider to Windows Azure without fetching the contents first and saving them as blobs in Windows Azure. Another use case for this feature is that now you could easily use Windows Azure Storage as a backup of your blobs data stored in another cloud provider storage.
- Please note that you will be charged for data egress if your source and target storage accounts are in different data centers.
More details about this can be found here: http://blogs.msdn.com/b/windowsazurestorage/archive/2012/06/12/introducing-asynchronous-cross-account-copy-blob.aspx.
Improved Lease Functionality on Blob Resources
Essentially lease functionality on blobs allow you to acquire an exclusive lock on a blob to prevent it from being modified or deleted. Yet another use case for lease functionality is to manage concurrency in a multi-threaded environment. Steve Marx has written an excellent blog post about this which you can read here: http://blog.smarx.com/posts/managing-concurrency-in-windows-azure-with-leases.
In this version, the lease functionality has improved significantly. Here are some of the improvements done:
- Lease operations can now be performed on a blob container. Earlier you could only do it only on blobs.
- You can now acquire a never-ending lease. Earlier you could only acquire lease on a blob which was valid for a maximum of 1 minute only (this 1 minute limit was hard-coded. You can’t change that in previous version). Apart from acquiring a never-ending lease, you can also acquire short-spanned lease valid from 15 – 60 seconds (thus more granular control to you).
- You can now define your own lease id (in GUID format) when acquiring lease. Earlier the lease id was created by Windows Azure Storage and returned to you. Furthermore, you can change the lease id value on an active lease.
- You can provide a time period up to which a lease should continue to remain active when breaking an acquired lease.
More details about this can be found here: http://blogs.msdn.com/b/windowsazurestorage/archive/2012/06/12/new-blob-lease-features-infinite-leases-smaller-lease-times-and-more.aspx
Shared Access Signatures for Table Resources (Again a Very BIG Yay!!!)
Till now you would need be a storage account owner to perform operations (like CRUD operations on entities) on table resources. NOT ANY MORE!!! With the latest releases, Windows Azure Storage has extended the Shared Access Signature functionality to table resources which was only available for blob resources till now.
A Shared Access Signature (SAS) functionality provides granular access to various storage resources. Essentially the way it works is that you configure a SAS token with predefined permission (read, write, list, delete etc.) which then appears as a part of what is called a signed URL. A user in possession of this signed URL is allowed to perform operations allowed by the SAS token using REST API without actually have access to storage account credentials.
A few comments about SAS for table resources:
- You can grant access to either an entire table (i.e. all data in the table) or a table range (i.e. selected data in a table). Let’s say you’re building a multi-tenant application and you are storing each tenant’s data in a separate table, you can now use SAS functionality to create tokens for individual tables and give a signed URL for each table to concerned tenant if they need access to raw data. If you’re developing a multi-tenant application where data for all your subscribers is going in one table, you could generate a SAS token on a table range and then give signed URL for each range to concerned tenant if they need access to raw data.
- You can grant access rights to the specified table or table range such as Query (r), Add (a), Update (u), Delete (d) or a combination of them. What this means is that you could have really fine-grained control over what a user who’s in possession of this signed URL could do with the data.
- You can either define a start and end time to provide time limited access or provide access to very large time period.
- Like with SAS for blobs, you can generate SAS tokens with or without an access policy. It is recommended that you generate SAS tokens with access policy as managing them becomes much easier.
More details about this can be found here: http://blogs.msdn.com/b/windowsazurestorage/archive/2012/06/12/introducing-table-sas-shared-access-signature-queue-sas-and-update-to-blob-sas.aspx
Shared Access Signatures for Queue Resources (Again a Very BIG Yay!!!)
Till now you would need be a storage account owner to perform operations (like CRUD operations on messages) on queue resources. NOT ANY MORE!!! With the latest releases, Windows Azure Storage has extended the Shared Access Signature functionality to queue resources as well (along with table resources) which was only available for blob resources till now.
A few comments about SAS for table resources:
- You can grant access rights to a specified queue such as Read or Peek at messages (r), Add message (a), Update message (u), and Process message (p) or a combination of them. What this means is that you could have really fine-grained control over what a user who’s in possession of this signed URL could do with the data.
- You can either define a start and end time to provide time limited access or provide access to very large time period.
- Like with SAS for blobs, you can generate SAS tokens with or without an access policy. It is recommended that you generate SAS tokens with access policy as managing them becomes much easier.
More details about this can be found here: http://blogs.msdn.com/b/windowsazurestorage/archive/2012/06/12/introducing-table-sas-shared-access-signature-queue-sas-and-update-to-blob-sas.aspx
Improved Shared Access Signature for Blob Resources
Shared Access Signature functionality for blob resources has been there for a long time with Windows Azure. As mentioned above, you could generate SAS tokens for a blob resource with or without an access policy. That feature is still here. What has changed is the duration for a SAS token’s validity when generated without an access policy. Earlier when you created a SAS token without an access policy, it was only valid for 1 hour only. In the new version, this restriction is removed. What this means is that now you can provide SAS tokens that are created without an access policy which are valid for longer durations (than 1 hour limit imposed by previous version).
More details about this can be found here: http://blogs.msdn.com/b/windowsazurestorage/archive/2012/06/12/introducing-table-sas-shared-access-signature-queue-sas-and-update-to-blob-sas.aspx
API Changes
Obviously to support these changes, API has been modified. Following is the list of new or updated operations in REST API:
Copy Blob
Copies a blob from one blob container to another blob container either in the same storage account or another storage account.
Documentation Link: http://msdn.microsoft.com/en-us/library/windowsazure/dd894037
Lease Blob
Acquires an exclusive lock on a blob to prevent any write/delete operations on them.
Documentation Link: http://msdn.microsoft.com/en-us/library/windowsazure/ee691972
Abort Copy Blob
Aborts a pending Copy Blob operation and leaves the destination blob with zero length and full metadata.
Documentation Link: http://msdn.microsoft.com/en-us/library/windowsazure/jj159098
Lease Container
Acquires an exclusive lock on a blob container to prevent any write/delete operations on them.
Documentation Link: http://msdn.microsoft.com/en-us/library/windowsazure/jj159103
Get Table ACL
Returns information about access policies defined on the specified table which can be used to create SAS tokens.
Documentation Link: http://msdn.microsoft.com/en-us/library/windowsazure/jj159100
Set Table ACL
Sets access policies on the specified table which can be used to create SAS tokens.
Documentation Link: http://msdn.microsoft.com/en-us/library/windowsazure/jj159102
Get Queue ACL
Returns information about access policies defined on the specified queue which can be used to create SAS tokens.
Documentation Link: http://msdn.microsoft.com/en-us/library/windowsazure/jj159101
Set Queue ACL
Sets access policies on the specified queue which can be used to create SAS tokens.
Documentation Link: http://msdn.microsoft.com/en-us/library/windowsazure/jj159099
Summary
As I mentioned before, the changes in quantity may not be large however the impact these changes would create would be significant. I’m really impressed with these changes and can’t wait to give them a try.
I hope you have found this information useful. If you think I have made some errors and provided some incorrect information, please feel free to correct me by providing comment. I’ll fix those issues ASAP.
Stay tuned for more Windows Azure related posts. So Long!!!
