vSphere 4.1 Array Integration on steroids

We've been anxiously waiting for VMware's announcement of vSphere 4.1 for weeks.  There are many big things in this release, including significantly scaling the management capabilities of vCenter and increasing the number of simultaneous vMotions that are supported.  The door is open for ESX deployments to achieve much greater densities than they could previously and that's a big deal to large enterprises who want to get more resources under the control of fewer points of management. There are still great gains to be made in consolidation - more on that later.

In the storage world, there are a couple big things, SIOC and array integration through the VAAI API.  Technodrone has put together an excellent post on SIOC and I highly recommend that anyone wondering how to make this functionality works should go to this post and read it. Array integration has been advanced in three ways:

• Hardware assisted locking
• Full copy
• Block zeroing

Array integration through the VAAI API is already at a very advanced status at 3PAR with some of the most important functions implemented through our I/O co-processor ASIC.  While some companies want to write off the importance of hardware, 3PAR believes there are many things that need to be done in hardware to get the performance needed to truly scale storage for virtual environments.  Our co-processors are key to getting much greater storage utilization and higher VM ratios and are one of the 3PAR innovations that separate our best of breed products from everybody else.  The capabilities discussed below are available in the hardware today, and will be enabled with a software upgrade in September.

OK, lets talk about hardware assisted locking first. For customers that have experienced locking problems, this is a big deal. The problem has been well-documented online - but in a nutshell, customers have run into problems where an operation that locked the LUN for a VMFS did not complete, thereby freezing all I/Os for all systems using that LUN. That was certainly a nasty problem - not a bug necessarily, but certainly an incredible pain in the rear to all involved.

VMware's response in vSphere 4.1 was to include a command in the VAAI API using an atomic test and set instruction for implementing fine grained locks for small block sizes. There will still be locking in VMware, but on a much smaller scale.

Unique to 3PAR is the fact that this new locking mechanism is implemented in our I/O co-processors where it completes very quickly, as opposed to implementing it in code in the controller. If you consider an environment with high VM ratios and multiple vMotions going on you want this granular locking mechanism to as quickly as possible. Nobody else comes close to the speed that 3PAR processes them.

Next is the new Full Copy capability - also with co-processor assistance to reduce the capacity of the copy that is made. 3PAR has zero detect and reclaim technology integrated into the co-processor. With zero detection running in an array, as new writes are made, strings of zeros are detected by the co-processor and those blocks are returned to free space inside the array.  If future reads are made to those blocks, zero values are returned, but not from disk. The result is that copies of VMDKs with lots of zeros in it, will be much smaller after the copy is made - and the copy will proceed much faster.

This sort of functionality works amazingly well with EagerZeroThick (EZT) volumes in vSPhere.  VMware requires EZT for Fault Tolerance (FT) and MSCS clusters and also recommends EZT for high performance. The main complaints about EZT are that it takes extra time to write all those zeroes when the VMDK is created and that it doesn't work well with thin provisioning. With 3PAR's zero detection, the time it takes to writes all those zeros and the space they consume is a non-issue, but more on that later.  Virtual Geek at EMC wrote about VAAI today and in his discussion of what does not work for full copy he mentioned copying from an EZT volume to a thin provisioned one.  Actually, he's wrong about that where 3PAR is concerned, because EZT to Thin works very well on a 3PAR array with zero detect.

The image below illustrates the advantages of using EZT on a 3PAR array with zero detection:

The last API element to discuss is Block Zeroing. The idea is that the host communicates to the array to write a string of zeros when it is provisioning storage or overwriting blocks to a non-EZT VMDK.  vSphere writes a lot of zeroes in order to maintain data integrity with multi-tenancy. The hypervisor zeroes zero out blocks prior to writing them in order to ensure that a virtual data imprint from an old VM does not occur for a new VM.

But writing all those zeroes consumes CPU and I/O bandwidth that could actually be used productively, so VMware included a new command to offload the host from writing zeroes, effectively shunting that workload to the array. Voila - problem solved with 3PAR!!  The zero detection and reclamation technology in a 3PAR array not only offloads the host from writing zeroes, but it also gives customers instantaneous reclamation of capacity with a smaller digital footprint (less capacity consumed) and faster performance. That's pretty cool and it's a trifecta that only 3PAR  has.

What is amazingly cool about today's vSphere announcement for 3PAR customers is that all three API elements, hardware assisted locking, full copy and block zeroing are already implemented in 3PAR's T and F series hardware platforms, and will be usable by the end of September with a firmware upgrade.

Our co-processor architecture really delivered for us this time. But it's been delivering the goods for our customers for a long time already. In virtualized environments our customers tell us they double their VM density, while cutting their storage capacity in half - all while reducing the amount of storage administration necessary by 90%. Those stats can be hard to believe, but when you look at what we delivered on the first day vSphere 4.1 was announced - when most people didn't even know we were working on it - it might make it easier for people to understand why. 

We take virtualized environments very seriously. People that don't know about 3PAR don't consider us to be a leader in virtualization, but when they find out the depth of technology we have and how well it works across our entire product line they understand we are leading in ways that really pay off for them. And the bigger they are, the bigger the rewards can be - especially after today.

Comments

[Anonymous]

Amazing. As a data center architect I am impressed.

[Robert Weilheim]

Way to go 3PAR! Great investment protection for your customers. I hope you have a big update to your vCenter plug-in coming too..could use some feature enhancements to keep up with your competition.

[Robert Weilheim]

In Chad's article he shows in green all of the EMC integration points with the vSphere stack. Can you comment on where all 3PAR integrates? For example, I think only EMC had a multi-pathing plug-in. It would be sweet if 3PAR matched EMC one for one.

[Roggyblog]

Thanks for this, nice write up, any chance of a video preview before September?

[Chris Fricke]

Nicely done guys! I haven't researched your major competitors on these features yet but it's nice to see the industry is getting it. I've whined for years on the importance of reclamation with thin provisioning (I'm sure I'm not alone) and this zero block detection and dynamic reclamation for VMware is huge. Is there a road map for extending this concept to general storage volumes?

Think about it: Any volume with dynamic inline zero block management/reclamation in the hardware coupled with seamless caching, tiering and deduplication... it's almost sexual.

[Visiotech]

The way I like it...well done Mark!

[Anabolic Steroids]

Way to go 3PAR! Great investment protection for your customers. I hope u have a big update to your vCenter plug-in coming too..could use some feature enhancements to keep up with your competition..

[Derek]

Check out my block zeroizing performance tests here:

http://derek858.blogspot.com/2010/12/3par-vaai-write-same-test-results-upto.html