StorageRap

From viral spam to virulent sham

The twitterverse is busy again today with discussions surrounding EMC's us of spambots to generate views of videos they are trying to make viral.  If you are interested in seeing what is being said, check out these people's tweets and you'll be off on a trip down a dark hole. 

@johnful,    @dvellante,    @sfoskett,    @valb00,    @furrier

Here are a couple cartoons I made about it last week from my new cartoon, Ineption:

Netapp's Val Bercovici suggest this viral spamming as the end of innocence in social media, but innocence exited the social media stage long ago. 

I'm much more concerned about how large companies like EMC can use social media to suggest product and customer relationships that stretch the truth well beyond the impressions that a reader might take away from reading suggestive blog posts from respected corporate voices. As "unofficial company statements" that are more influential than press releases, social media pieces can distort things in a way that more-accountable corporate marketing are not allowed to.

Last week, Chad Sakac and Chuck Hollis published blog posts that pointed to an EMC white paper about details of a VMAX implementation at Terremark, an excellent 3PAR customer.  Readers of these posts would probably think that VMAX was being used as the storage behind Terremark's multi-tenant,  Enterprise Cloud service offering.  That would be stretching things more than just a little bit.  I commented on both blogs and the responses to my comments were interesting.  I guess I feel a little kinder towards Chad as a result. 

It is possible that somewhere in the world, a VMAX is being used by Terremark.  One would expect Terremark to be looking at various storage platforms as a matter of course, it only makes sense for them. After all, VMware made a significant investment in Terremark last year and we all know who owns VMware. There are certain favors that EMC can ask that vendors such as 3PAR can't. But Terremark also has to operate Enterprise Cloud in their US major data centers every day and the storage they use for that is not in a test lab - it's production - and it is 3PAR storage. 

And its not for lack of trying on EMC's part.  Last November when VCE was announced, Terremark was discussed as a featured customer in both Chad's and Chuck's blogs.  That was OK,  I understand the excitement that surrounds a big announcement.  But nine months later, to suggest that this announcement had given birth to a major production environment for a service that it is not supporting sort of stuck in my craw.

Here is a video I made at VMworld with Jason Lochhead, CTO of Hosting Solutions at Terremark last year where he talks about vCloud Express and Enterprise Cloud.  Very cool offerings and definitely on the leading edge of VMware-based service offerings.

It's not a viral video, but it has a lot more to say about what people care about than the videos EMC has been chasing with spambots.

The Netapp vSphere 4.1 decoder ring

 
There was a lot written last week surrounding VMware's release of vSphere 4.1. Netapp appeared to have a lot to say, but it was confusing to figure out what they were really talking about.  I think I've got it now.

It's unusual for a company to be invited as a centerpiece of high-visibility festivities and then mysteriously decide not to follow through. It would be like getting complimentary tickets and backstage passes from Lady Ga Ga herself, telling all your friends about it and then not going. It it does make one wonder. Why wouldn't you do whatever it takes to be included in VMware's big summer announcement party? Well, if you're Netapp, the answer appears to be - "Being there is over-rated. Just make sure everyone thinks you were."  Call it Photoshop for PR or call it keeping your poker face, it's a mash up of a blown opportunity and opportunistic courage. 

The excitement for VMware's storage partners was concentrated in two areas: VAAI (vStorage API for array integration) and SIOC (Storage I/O Control).  The initial release of VAAI includes new SCSI block storage commands that allow the arrays to offload host systems from redundant, resource-consuming tasks. SIOC is a method for managing I/O queues to create more fairness in accessing storage resources. Netapp issued a press release last week in conjunction with the vSphere 4.1 release, but it was for their Virtual Storage Console, not for the support of the storage enhancements in vSPhere 4.1. There was a flag waving mention of VAAI:

"Additionally, NetApp is supporting the new VMware vStorage APIs for Array Integration (VAAI) capabilities that offload data management tasks from the host server to the storage system. This can free up host CPU cycles for better performance and increased virtual machine density." 

That's not exactly saying anything, but its more than they had to say about SIOC, which was zilch. 

The bottom of the release directs readers to Vaughn Stewart's blog for more info.   Apparently, Netapp's PR department left the rest of the innuendo up to Vaughn - a diligent and loyal Netapp employee who understands that sometimes a vendor blogger doubles as a  PR bagman.  It looks like I need to add a new chapter to  Vendor Blogging with Dummies.

You have to dig into the comments to get some of the details, but Vaughn's blog does a decent job explaining that Netapp is working on delivering VAAI functionality in Q4 2010.  Now, that's not all that late considering its only 6 months or so away, but as a privileged insider to VAAI development, it's not a great showing either. In fact, it wouldn't surprise me if some of the companies who were not in the program, such as Compellent, HP, IBM and Xiotech come out with VAAI plug-ins before Netapp does. As for 3PAR, we will have our VAAI plug-in available in September as part of a maintenance release. We didn't have a lot of time to develop VAAI functionality after gaining access to the APIs in early 2010, but we fast-tracked the development of it in order to make the announcement.  

As much as I admire Vaughn's hutzpah for stepping in to carry the load that others at Netapp should have, there were a few problems with what he said. First was the absurd statement that " SAN is attempting to be more NAS-like". There is so much wrong with that statement that it's difficult to find a place to start.  Who or what is SAN? Is VMWare SAN? Is the T10 SCSI standards committee SAN? Is SAN the being an embodiment of SAN the block protocol?  Is there a virtual reality thing going on here? And what is NAS-like anyway? Does it have anything to do with the size of one's beak or the way particular vowels resonate in the sinus cavities? Or is it like racing the back roads in a used chevy? Whatever Vaughn meant, I tend to dislike the imprecision of technology anthropomorphism.   

The second thing Vaughn said was  "As for the first release of VAAI... These features ALREADY EXIST in NFS."   Really, block zeroing?  That is a function developed for EagerZeroThick volumes, which are only supported on VMFS datastores, not NFS datastores. Perhaps we will see that change in the future, but for now its SAN only.

Hardware assisted locking is a way to allow smaller granular locking for VMFS and addresses an issue with VMDK-level operations in a shared datastore. Because NFS puts VMDKs in separate datastores, which are locked independently, hardware assisted locking is unnecessary for NFS. In other words, its a SAN only function because the current NFS datastore architecture doesn't need it.

The other API in VAAI is Full Copy. This VAAI API appears to be functionally equivalent to a Netapp utility called RCU (Rapid Cloning Utility) that was included as a function in their Virtual Storage Console. It is not, however, something that exists in NFS, unless Netapp wants to give that feature to all it's NAS competitors.  As a vSphere function, Full Copy will be available to all vendors that implement the VAAI APIs.  It will be interesting to see what differences there are as far as programmatic control using the VAAI plug-ins, vendor-specific consoles and Powershell. 

Robert Cockerill from Thames River Capital: A man with broad responsibilities and 3PAR storage

3PAR customers like that fact that 3PAR arrays are so easy and fast to manage. In this video, Robert Cockerill from Thames River Capital in London talks about all the various things he does, his Windows-based infrastructure, how 3PAR's thin provisioning helps him manage it all and how simple it was to protect it with 3PAR Remote Copy.

Clarifying VAAI capabilities and implementations

Virtual Geek and I had a discussion on his post yesterday about vSphere's VAAI capabilities announced yesterday.

I wrote about the fact that we already had zero detect technology in our product, which is useful for the new Full Copy command because it allows customers to remove zeroed data from clones when they are created and return them to array free space.

The discussion became a bit confused when Chad interpreted what I was saying as pertaining to Block Zeroing.

Block Zeroing and Full Copy are different aspect of the VAAI API.  The intent of block zeroing is to reduce the amount of CPU effort and storage traffic required to write zeroes across an entire EagerZeroThick (EZT) VMDK when it is created.  The intent of Full Copy is to make clones of VMs quickly without consuming I/O bandwidth.   Things get interesting when you start thinking about making a full copy of an EZT VMDK that was created using VAAI with block zeroing - but I'll discuss that later.

I also want to clarify what zero detection technology is. 3PAR T and F class arrays have zero detection technology, which is enabled by Thin Persistence software, that recognizes zeroed blocks as they are read by the array and returns them to the array's free pool.  Any read requests made to these block addresses will return a zero value. In essence it is dedupe for zeroes.

However, Zero detection is not needed when an EZT VMDK is created using the VAAI plug-in because the array will recognize the intent of the command and not write the zeroes. In other words, the VMDK will only contain a very small amount of reserved space when it is created. Again, any attempts to read blocks in those ranges will return zero values. Zero detection is effectively bypassed during the creation of the EZT VMDK.

The exception to this behavior is when the EZT VMDK being created is written to a thick volume - in that case the array will write zeroes across the entire VMDK. 

The remaining cases for the creation of EZT VMDKs on 3PAR arrays occur when the VAAI is not used. For a thick volume, the entire VMDK has zeroes written to it.  Thin volumes not using zero detect also have zeroes written over the entire VMDK. Thin volumes with zero detect will not have zeroes written to them and will contain only a small amount of reserved space. 

FWIW, the reserved space is used as instantly-available capacity that can be allocated on-demand when writes start coming into the volume.  3PAR arrays always "read ahead" free space to improve the performance of thin provisioning.

The next bit here could be a bit thorny, so clear your head. The matter of making a Full Copy of an EZT VMDK to a thinly provisioned volume was something Chad said was not allowed.  My assumption here is that the type of thin provisioning used makes a big difference.

For instance, if you are using TP from VMware, I could see where they would not allow a full copy to be made. The problem is that the full copy will return all the zero values for the source VMDK, whether or not those zeroes were ever actually written - and write them to the target TP volume.  In other words, the target could be much larger than the source. In the VMware TP scheme, this could make for problems in a hurry if you were making a bunch of clones this way.

In contrast, if you were using a 3PAR array with zero detection, the Full Copy of the source VMDK would return zeroes for the entire VMDK, but the zero detection would strip them out again as the target was being written.  You could make as many clones as you wanted this way, knowing that the physical capacity they consume would be a multiple of the physical capacity consumed by the source VMDK. In other words, you wouldn't have to worry about virtual zero bloat making a mess of your VMFS volume.

One of the big differences between 3PAR's zero detection technology and other vendors zero-reclaim technology is that 3PAR's process is real-time-on-ingestion as data comes into the array, whereas zero-reclaim works in a post processing fashion after the zeroes have already consumed disk space. This could be a significant difference in many cases because the post-processing method has the potential to create unexpected capacity-full conditions before the zero-reclamation process even has a chance to start.

vSphere 4.1 Array Integration on steroids

We've been anxiously waiting for VMware's announcement of vSphere 4.1 for weeks.  There are many big things in this release, including significantly scaling the management capabilities of vCenter and increasing the number of simultaneous vMotions that are supported.  The door is open for ESX deployments to achieve much greater densities than they could previously and that's a big deal to large enterprises who want to get more resources under the control of fewer points of management. There are still great gains to be made in consolidation - more on that later.

In the storage world, there are a couple big things, SIOC and array integration through the VAAI API.  Technodrone has put together an excellent post on SIOC and I highly recommend that anyone wondering how to make this functionality works should go to this post and read it. Array integration has been advanced in three ways:

• Hardware assisted locking
• Full copy
• Block zeroing

Array integration through the VAAI API is already at a very advanced status at 3PAR with some of the most important functions implemented through our I/O co-processor ASIC.  While some companies want to write off the importance of hardware, 3PAR believes there are many things that need to be done in hardware to get the performance needed to truly scale storage for virtual environments.  Our co-processors are key to getting much greater storage utilization and higher VM ratios and are one of the 3PAR innovations that separate our best of breed products from everybody else.  The capabilities discussed below are available in the hardware today, and will be enabled with a software upgrade in September.

OK, lets talk about hardware assisted locking first. For customers that have experienced locking problems, this is a big deal. The problem has been well-documented online - but in a nutshell, customers have run into problems where an operation that locked the LUN for a VMFS did not complete, thereby freezing all I/Os for all systems using that LUN. That was certainly a nasty problem - not a bug necessarily, but certainly an incredible pain in the rear to all involved.

VMware's response in vSphere 4.1 was to include a command in the VAAI API using an atomic test and set instruction for implementing fine grained locks for small block sizes. There will still be locking in VMware, but on a much smaller scale.

Unique to 3PAR is the fact that this new locking mechanism is implemented in our I/O co-processors where it completes very quickly, as opposed to implementing it in code in the controller. If you consider an environment with high VM ratios and multiple vMotions going on you want this granular locking mechanism to as quickly as possible. Nobody else comes close to the speed that 3PAR processes them.

Next is the new Full Copy capability - also with co-processor assistance to reduce the capacity of the copy that is made. 3PAR has zero detect and reclaim technology integrated into the co-processor. With zero detection running in an array, as new writes are made, strings of zeros are detected by the co-processor and those blocks are returned to free space inside the array.  If future reads are made to those blocks, zero values are returned, but not from disk. The result is that copies of VMDKs with lots of zeros in it, will be much smaller after the copy is made - and the copy will proceed much faster.

This sort of functionality works amazingly well with EagerZeroThick (EZT) volumes in vSPhere.  VMware requires EZT for Fault Tolerance (FT) and MSCS clusters and also recommends EZT for high performance. The main complaints about EZT are that it takes extra time to write all those zeroes when the VMDK is created and that it doesn't work well with thin provisioning. With 3PAR's zero detection, the time it takes to writes all those zeros and the space they consume is a non-issue, but more on that later.  Virtual Geek at EMC wrote about VAAI today and in his discussion of what does not work for full copy he mentioned copying from an EZT volume to a thin provisioned one.  Actually, he's wrong about that where 3PAR is concerned, because EZT to Thin works very well on a 3PAR array with zero detect.

The image below illustrates the advantages of using EZT on a 3PAR array with zero detection:

The last API element to discuss is Block Zeroing. The idea is that the host communicates to the array to write a string of zeros when it is provisioning storage or overwriting blocks to a non-EZT VMDK.  vSphere writes a lot of zeroes in order to maintain data integrity with multi-tenancy. The hypervisor zeroes zero out blocks prior to writing them in order to ensure that a virtual data imprint from an old VM does not occur for a new VM.

But writing all those zeroes consumes CPU and I/O bandwidth that could actually be used productively, so VMware included a new command to offload the host from writing zeroes, effectively shunting that workload to the array. Voila - problem solved with 3PAR!!  The zero detection and reclamation technology in a 3PAR array not only offloads the host from writing zeroes, but it also gives customers instantaneous reclamation of capacity with a smaller digital footprint (less capacity consumed) and faster performance. That's pretty cool and it's a trifecta that only 3PAR  has.

What is amazingly cool about today's vSphere announcement for 3PAR customers is that all three API elements, hardware assisted locking, full copy and block zeroing are already implemented in 3PAR's T and F series hardware platforms, and will be usable by the end of September with a firmware upgrade.

Our co-processor architecture really delivered for us this time. But it's been delivering the goods for our customers for a long time already. In virtualized environments our customers tell us they double their VM density, while cutting their storage capacity in half - all while reducing the amount of storage administration necessary by 90%. Those stats can be hard to believe, but when you look at what we delivered on the first day vSphere 4.1 was announced - when most people didn't even know we were working on it - it might make it easier for people to understand why. 

We take virtualized environments very seriously. People that don't know about 3PAR don't consider us to be a leader in virtualization, but when they find out the depth of technology we have and how well it works across our entire product line they understand we are leading in ways that really pay off for them. And the bigger they are, the bigger the rewards can be - especially after today.

10 common misconceptions about sub-volume tiering

1.  Everybody has it.

2.  All implementations are the same.

3.  Compellent has the most full-featured implementation.

4.  It will use expensive SSD resources for low-priority apps.

5.  It won't work for VMFS data.

6.  EMC is shipping it.

7.  Tiering+SATA+SSDs will eliminate FC/SAS drives.

8.  It only works with SSDs.

9. You will end up thrashing data across tiers.

10. You have to buy it before you can determine if it will work for you.

3PAR's sub-volume tiering, Adaptive Optimization (AO), comes with a lot of features that make it much better than what competitors say. It doesn't require SSDs and it's not for everybody, but if you need it, it will do the job.    

Is there a storm brewing around HP's XSP/cloud business?

The news here in Silicon Valley this morning is that HP is planning to overhaul its Enterprise Services Division by laying off 9000 employees and developing a business in automated XSP/cloud data center services.

When HP announced its almost $14Billion EDS acquisition back in 2008, Nicholas Carr and others wrote about how HP needed an IT outsourcing services business to keep up with IBM.   Others, such as Om Malik, suggested the move was designed to bring HP into the XSP/cloud computing age. 

Both camps were probably correct. When it got into the outsourcing business with EDS, HP undoubtedly was hoping for a cash cow business like IBM's, but in the last couple years, the XSP/cloud industry has risen as a serious economic alternative to outsourced services. So what can HP do?  Try to milk the outsourcing business as much as it can, while it simultaneously tries to take a position in the XSP/cloud services business.  A couple years ago it looked like a way to hedge their bets, but the either/or scenario doesn't always pan out the way you think it will and it looks like HP has to make a change in order to optimize its EDS investment. It's not easy changing horses, but that's what you have to do sometimes.

The most relevant aspects of this development in my opinion could be:

• HP could become a competitor to a large number of it's current XSP/cloud services customers, many of whom might decide to turn to other vendors for their servers if they feel threatened by HP's XSP/cloud business. This could actually help HP's new competitor Cisco sell it's new UCS servers to customers that would otherwise be loyal to HP. If there is a customer backlash, HP's servers will probably have to be either cheaper or more technically advanced than its competitors - neither of which are very likely considering the maturity of the technology. HP would also like to increase its networking and storage sales to these customers, but they will both be dead in the water if they perceive that HP is encroaching on their turf.
• Assuming the 9000 layoffs will come from employees providing IT outsourcing services, it will likely mean HP plans on migrating many of its outsourcing customers to their XSP/cloud services business. But customer migrations tend to become low hanging fruit for competitors. The dirty little secret in the IT business is that profits are often made on the backs of competitors' mistakes. If a feeding frenzy develops around HP's outsourcing customers it could turn out badly for HP.
• It appears that HP intends to create the infrastructure for their XSP/cloud services using their own Converged Infrastructure products.  This would give them certain cost advantages for acquiring and installing products in their XSP/cloud data centers. The risk is that HP's infrastructure could have certain shortcomings due to certain limitations of it's own products. If HP's XSP/cloud service competitors use best of breed products and HP struggles to keep up with any of its own products, it could create a tipping point for HP's XSP/cloud customers who want - but can't get - the same service levels from HP as they can from competitors. 

A number of 3PAR XSP/cloud customers are running HP servers in an environment we call 3CV (3PAR storage, HP servers and VMware hypervisors).  It's a stable, scalable, best of breed environment that our customers are very happy with.   HP, the company that makes the servers for this solution could create pressures in the market stemming from their EDS acquisition that have ripple effects on the rest of their ecosystem. If it happens, it happens, and we'll adjust along with our customers and the rest of our partners.  It's all part of what makes this technology business so blasted interesting. There's always something.

Video demo of 3PAR's Plug-in for VMware vCenter and VMware Site Recovery Manager

Here is a video demo prepared by Maneesh Jain, one of 3PAR's Solutions Architects, of 3PAR's Plug-in for VMware vCenter and 3PAR's VMware Site Recovery Manager.  The Recovery Manager demo shows not only how to recover a VM, but also individual files within a VM. Enjoy!

VPLEX undressed

OK,  Monday's post lacked the punch that people have come to expect from me where EMC announcements are concerned.  Thanks to my readers who were disappointed and told me so. This post is for you. 

 

VPLEX (announced Monday with all the hype that EMC could muster) is the result of EMC trying to become something more than a storage company. They put a systems professional in charge of developing storage products and produced a science project that looks very attractive as a technology but relegates storage to a secondary role and questions EMC's commitment to solving storage problems. In fact, VPLEX probably creates more storage problems than it solves. More on that later.

The concept of teleportation for virtual machines resonates with childhood space fantasies. I remember summer afternoons from my childhood going out to a nearby field with my brother and his model rocket club. My brother built some stunningly cool looking model rockets, and the amazing thing was that they all went where he intended them to - up. Just as amazing was the fact that the engine always ejected the nose cone and parachutes to bring his creations back to earth gently.  It was a riot running across that field trying to catch them in our bare hands as they descended. A few years later those rocket designs turned into a set of World Book Encyclopedias that he won at a big science fair in Michigan.  I was prouder of him than any kid brother could be, even though I was just the rocket runner.

However, his rocket club also had a couple of less-talented rocketeers who also launched their contraptions with great fanfare and, quite often, spectacular disappointments. Engines became separated from the rockets holding them, shooting around at random on the ground - and then there was the memorable starship design that took a 90% turn after exiting the launch wire and proceeded to fly parallel to the ground at height of 4 to 5 feet, sending parents, siblings and friends scrambling for cover behind a shrub, pet or each other.

It's good to believe in some things and to be wary of others.

 

So what is VPLEX anyway?

For starters, VPLEX is a virtualization appliance - a bump-in-the-wire product. EMC says it will take the technology at some point in the future and build it into their arrays, but like a lot of things, its much easier to talk about it than to actually do it. The irony of the situation is that it VPLEX fills a storage virtualization role that is similar to IBM's SVC - a product that EMC bloggers eagerly lampoon. EMC is downplaying the similarity to SVC and are spinning VPLEX as a teleportation device for your virtual machines.  Of course, the image of teleportation they want you to think of comes from Star Trek, but sci-fi fans are familiar with the risks of teleportation too - just for fun, recall the problems Jeff Goldblum had with it in The Fly.

The point is, VPLEX is an appliance and therefore it is another thing to manage and pay for. In fact it is at least 4 more things to manage (two here and two there) and can be as much as 16 additional things to manage. Just in case that point was not clear, VPLEX is new stuff to manage in between your servers and your storage.  Like the Wizard of Oz, EMC would like customers to "Pay no attention to those things in front of your storage, - they are virtual and therefore invisible." In fact, it's so invisible that EMC claims you can use it with any vendor's storage, which means there is minimal integration with storage functions. EMC is calling VPLEX Storage Federation, and yet it doesn't integrate with storage on any meaningful level. That's not federation, it's virtualization.

Suddenly, the virtualization of all your storage through an appliance is a trivial matter. But of course it adds considerable complexity to an environment that is already complex enough. To put a finer point on it, EMC customers are familiar with the down and dirty disappointing reality of working with Virtual Provisioning, SRDF and change management. They should expect to pay for additional professional services to figure out how to do these things with VPLEX inserted.  

And then there is the matter of managing mixed workloads in virtual environments, something that is a problem for a lot of customers.  EMC's technique of isolating workloads on certain disk and cache resources doesn't work for highly leveraged VMFS's where there are many VMs running concurrently. When the workloads are all mixed together, there is no way to isolate them. VPLEX appears to be an answer to that problem by allowing customers to move VMs elsewhere - presumably where there are more resources available. In reality, those VMs are going to be moved to another environment with the same storage limitations as where they came from.  Do you really think putting a distributed write cache between servers and storage is going to solve workload balancing problems?  If so, the fix will only be temporary before the same problems are repeated. And that is the problem with VPLEX - it doesn't solve anything by itself, it only allows a problem to be moved elsewhere - presumably someplace where there is more headroom that is not being as heavily utilized.

Storage systems that were designed for utility computing and assuming there would be mixed workloads to deal with, such as 3PARs InServ storage systems, are much more effective at maintaining performance levels for virtual machines and allow customers to operate at much higher system and storage consolidation ratios.

EMC wants you to imagine all the things you might be able to do with VM teleportation. I'd say, imagine how much fun its going to be doing the capacity planning for a VPLEX installation. How much storage do you need in location A and location B to support two way VPLEX functionality for a collection of 80 VMs? As it is with most things, it depends on a number of variables including whether or not VPLEX is being used to facilitate disaster recovery. It would not surprise me if VPLEX ends up effectively capping storage utilization at 50% for customers that employ it. If the maximum utilization is 50%, what will your actual utilization levels be?  Something far less than that. 

Reducing consolidation ratios

In a nutshell, all systems that are fronted by VPLEX will have the same write cache data so that a read from those systems will be serviced from the write cache, if the data has been recently written. However, if a read request is made from a VM for data that is not in VPLEX's write cache, the data will be read from either a local or remote storage array. That's why there are distance limitations to VPLEX today - the time it takes to cross-ship data from a remote location can't be too obnoxious. 

It's also why EMC's Virtual Geek spent time talking about using applications that have a single writer - such as v-Motion. A VM with an application that attempts to write to the same block from two different locations is apparently problematic. My guess is that applications with long sequential reads might not be so great either.

That implies a couple things: You should expect to find out there are limitations as to what applications can be running with VPLEX. Maybe identifying VPLEX friendly applications will be easy, but perhaps this task will be harder than assumed. If so, the best answer may be to engage EMC's pricey professional services organization. The details, the details.

The other implication is that VPLEX will limit the number of VMs that can be running on a single vSphere server. After all, you wouldn't want to have any congestion problems forwarding write cache data through the VPLEX network. The higher your server consolidation ratio is, the more likely it is that you could run into bottlenecks of varying types. Expect to see best practices for VPLEX that scale back your server consolidation ratios.  

Just because you can do something, doesn't mean it's a smart thing to do.

Knobs, Prison Knobs and STACK WARS

Technology integration makes computing products much easier to use and significantly drives down the cost and effort of owning it. For instance, technologies such as WiFi that were recently beyond the grasp of most people are now inexpensively integrated into PCs and usable by almost anyone.

The trick with integration is understanding what variables should be exposed - or as my friend Rick Vanover likes to say - how many knobs there are to turn.  End user and infrastructure provider requirements differ considerably when it comes to knobs. For instance, Apple computers are great end user machines because they lack knobs, but are not always loved by technology professionals for the same reason. Data center operators need products with knobs in order to accommodate all the cross-purposed requirements that stretch beyond a one-size-fits-all design.

So knobs are generally good - but like so many things - their usefulness depends on how effective they are and their station in FARLEY'S HIERARCHY OF KNOBS, which includes the following levels:

Suicide Knobs:  knobs that delete data and make things blow up.  A good example of a Suicide Knob is something that formats storage.

Prison Knobs:  knobs that make changes that are very difficult or impossible to reverse.  Many storage provisioning knobs fall into this category. Once you provision and reserve storage with most storage arrays today you are stuck with that decision until the array's EOL.

Faux Knobs:  knobs that never seem to do anything, no matter how far you turn them.  For features past and future, but not now.

Random Knobs:  knobs that produce unanticipated results that can go unnoticed for years.  These are the knobs that fuel the technical publishing industry.

Slippery Slope Knobs: knobs that start you down a path to ruin through a chain of system dependencies. These are the knobs you spend a lot of money to learn about in vendor classes. 

Dumb Ass Knobs:  knobs that do things, but not anything useful. Granted there is a LOT of subjectivity in making a call on a dumb ass knob - but we all agree they exist.

Honest Knobs:  knobs that actually do something you need them to without having to plan for weeks on how to use them.  Most knobs should fall into this category, but alas!

Magic Knobs: knobs that do things so useful it makes you wonder how anybody thought of a knob like that. Most of these knobs are actually Honest Knobs, but we are so accustomed to seeing Suicide, Prison, Faux, Random, Slippery Slope and Dumb Ass knobs that we are blown away by a truly great Honest Knob.

Which brings me indirectly to the discussion of STACK WARS,  which, have been centered around the announcement of VCE (VMware\Cisco\EMC) and their vBlock concept last November. 

 

I'd like to say I was surprised yesterday when graphically-challenged Hitachi announced their intention to sell their own Unified Cloud Graphic, (complete with Hitachi compute servers!).  But it wasn't a big shock considering their marketing strategy of "just copy it".  

 

I really don't know how they expect their graphic to compete with vBlock's graphic, with all the color, multiple font sizes and graphics within graphics.

What's missing from the both stack graphics are the knobs that administrators use to get real work done.  Yes, knobs tend to be part of the underlying details, but to anybody that actually uses a product, they are very important details. The detail that C-level executives need to understand is that the stack does not have nearly the automation that is being promised today and that administrators will be doing a lot of work, turning the knobs that the stack provides. Again, it's not the number of knobs that matters, as much as it is the quality of those knobs.

Some people have speculated that the vBlock was a knob-less invention that originated in the board rooms of the VCE companies. Some have even suggested that it was the fallout after a failed acquisition bid by Cisco to acquire EMC.  I don't know if THAT's true, but there is some evidence that the engineering groups in the companies involved have been scrambling to put meat on the bone.

Maybe someday stacks will be the next big thing, but I don't see it playing out that way unless an awful lot changes in the underlying products that make up the stack. Here's my take on STACK WARS:

STACK WARS give everybody something to write about- me included, right now!

Bloggers that write about stacks have a chance of getting jobs with stack vendors. If you are out of a job, start a stack blog today and twitter your back-stack off!

Stacks are all about packaging. Stacks will be assembled and shipped together (presumably), which could make things easier if your goal is to streamline receiving.

Stack products, are actually more services than products. However, if you ever want to make configuration changes in your stack it might not be economically feasible. (Think gigantic FRUs) For example, there is not a lot of flexibility in vBlock's configurations.

Due to the limited configuration options, stack resources are not likely to be used very efficiently and the economic return on the investment will lag. However, EMC customers are already accustomed to low storage utilization levels - so poor utilization might not be THAT big a deal. Definitely a weird way to win a point, but I 'll concede it grudgingly.

The business advantage of integration should be much lower costs. However, the VCE companies all need to maintain their margins if they want to satisfy investors. It's not clear how they will be able to leverage the integration effort to reduce the cost of vBlock, but then again if STACK WARS turn into PRICING WARS for STACKS, things could get very interesting. IBM must be STACKING up something - after all Hitachi already beat them to the punch.

The C level view of stacks are that they smooth out purchasing and operations expenses by providing a smaller number of Purchasing Knobs (that would be a Faux Knob).  John Nash posted in his blog last week,"The Case for the vBlock":

What is interesting is that, usually, the higher up in an organization you are communicating the better the Vblock conversation goes. Remove the detailed technical questions and the value of the Vblock idea really shines. You get a known “product” from trusted sources. You get known costs today as well as known costs for future expansion. It greatly removes the risk from the organization with unknown infrastructure expenses.

There you have it, vBlocks will be sold from the top down by Cisco and EMC - companies that are good at selling from the top down, which will make it somewhat easier for the VCE companies to justify their price tag. But that won't make the price any easier to swallow. 

As Nash wrote, "remove the detailed technical questions and the value of the Vblock idea really shines."  That's like saying chapulines (fried grasshoppers) might appeal if Anthony Bourdain is talking about them on TV, but your own personal experience chewing and swallowing them might be different. I'm not talking about price here, I'm referring to the experience of running the vBlock. There is going to be a lot more involved than the knob-less graphics portray.

The weakest link in the vBlock chain today is EMC's contribution.  There are far too many Prison (provisioning) and Slippery Slope Knobs in EMC storage.  They aren't the only vendor with this problem, but they are the E in VCE.  Provisioning storage with a v-Max is the about the same as it was with a DMX - despite what EMC employees would have you believe. 

Prison Knob provisioning creates a lot of problems for customers as storage ages and as demands shift.  Once storage has been reserved for usage in an EMC system, it is pretty much bound to that purpose. 

My advice is to buy the products with the most Magic Knobs and avoid those with the most Prison provisioning Knobs. If you have ever felt trapped by a storage configuration that you couldn't live with or afford, you know what I'm talking about.  Magic Knobs are those that reduce the effort to manage and change storage, increase the efficiency of storage and provide the most versatility for all applications, workloads and multi-tenancy.

The SWCSA rides again: A smackback for Georgens' whacked fast smack

No, it's not a SWCSA rap, but it is a steering wheel cam, complete with a surprise ending- inspired by Stu @ at EMC.   

A conversation with Charlotte Melen of Comic Relief

The UK-based charity, Comic Relief, is once again using 3PAR storage as part of its IT infrastructure for raising millions of Pounds during it's nationwide Sport Relief campaign the weekend of March 19-21.  StorageRap spoke with Comic Relief's Web Technology Manager, Charlotte Melen recently about the intangible benefits of working for Comic Relief and what technologies they use for their donations platform.   Besides 3PAR, other key vendors include HP, Cisco, VMware and Oracle along with payment services from Paypal and RBS.

EMC takes dumbass award with denial of reality strategem: Secure Multi-Tenancy and Virtual Domains

EMC can't help themselves.  Given what appears to be a new corporate mandate to deny competitive threats in their massive propaganda machine, they look pretty stupid when one of those threats involves their new BFF Cisco and their trophy-acquisition raison d'etre,VMware.   

Yesterday, Netapp, Cisco and VMware made a planned, well-orchestrated announcement of their secure, multi-tenancy architecture.  Trust me - everyone in the industry knew it was coming and it was the main reason we chose to time our vSPhere and vCenter Server announcement for Monday this week. Chuck Hollis, EMC's StorageMonkeys poll winner, responded with an incredibly ignorant post that does not even mention the companies involved, nor does it link to any of the numerous online postings discussing it. For as many words as La Bombast put into this blog entry, you would think he would have at least mentioned Cisco in his discussion on network security. Granted, this announcement puts EMC in a compromised position, seeing as how their most strategic partners have chosen to develop an advancement in data center technology with one of their most dangerous competitors, Netapp.  Nonetheless, EMC's inability to deal with the reality of it underscores the difficulties they have holding an objective, honest and open public discussion.

I like what Netapp, VMware and Cisco have done with their Secure Multi-Tenancy architecture. The ability to segregate and align applications, data and IT resources for management purposes is a very big deal. It is also one of the key concepts in virtualization technology.  Sharing resources among multiple applications and users provides powerful economic leverage (doh!) and aligning security and access controls on virtual boundaries is not only desirable - it is essential.  Role-based administration is certainly nothing new, and bringing that into the virtual data center is a necessary evolution in the technology's development.

So what about 3PAR?  Been there, done that and continue to lead in utility storage infrastructure technologies. 3PAR developed our Virtual Domain technology more than two years ago that brought  role-based administration and firewalls to utility storage. It's also why we developed our Cloud-Agile program for utility computing service providers and designed ourCloud-Agile SECURED solution into it.  We have done many things over the years that have been ahead of their time and Virtual Domains was one of them. We're happy to see Netapp, Cisco and VMware shine a light on this important technology area.  Too bad for EMC that they missed on this one. Guess they have some 'splainin and fudding to do!

Check out our capacity guarantee -
the program EMC doesn't want you to know about.

3PAR Countdown: New Plug-Ins for vSphere and vCenter Server

The news is here:

Curtis waves goodbye - but wait, last week was Thanksgiving - not Christmas!

W Curtis Preston follows his own path - always - and that is probably the main reason lots of storage geeks read his blog.  However, his unorthodox approach sometimes leaves me scratching my head, as it did today when he wrote a year end blog post on the last day of November, Data protection and data backup trends in 2009.  (Note to Curtis - There is still another month to go this year.)

Topics covered include companies and services that lost data, went out of business or had their futures prematurely stopped by corporate decree, so its not just saying goodbye to 2009, but to products and services that we'll never see again too. Will we need an addendum for the month of December? Stay tuned to Curtis' blog and find out.