StorageRap

Who's Waldo?

Earlier this week I posted a LOSA video about Hitachi's un-thin dynamic provisioning.  Towards the end of that mangatartoon, there are several masks shown. Two of them are death masks of famous people. 

Your quest is to declare who they are.  Bu wa ha ha ha ha

Ode to a pissing contest on twitter

I don't know about you, but I don't like it very much when vendors lock horns in Twitter.  I don't care if they want to go at it in their blogs but twitter space is a shared resource. The commons turns into a landfill in a hurry if people can't resist the urge to spit at each other. Anyway, I wrote a sonnet about the whole thing.  (mf note 2-27-09  Steve Todd's lovely comment inspired me to recast the final line)

Oh Twitter, stunted mouthpiece to a world
Of kindred internet souls and lurkers
We shoot off our tweets to whomever stirs
From gadgets employers give their workers
Short writs, clever quips and polite LOLling
Brief text treats like sweet chocolate kisses
They live for mere minutes before rolling
Down the shadowy slopes of the abyss

Alas, acrimonius tweets – a foul stench
Competing twitterature floods the channel
Trashing the commons of our workbenches
With corporate arguments most banal
No denying there’s passion in pissing
But more interesting tweets go missing
While most splendid tweets unheard, are missing

Fred Moore on Getting Rid of Data

I've always liked Fred Moore's analysis of our industry.  Here is his latest writ on the Wikibon site where he talks about thin provisioning and dedupe:

In traditional storage provisioning, application teams guess at how much storage they might consume and reserve that full amount on day one. The amount reserved can be used only by that application and is not available to others. This means that within a data center, large portions of costly storage go unused, and consume power and cooling resources even though no data is actually written to the disk.

That pretty much nails it.  Not all thin provisioning implementations are equal, so look into the details of each vendor that you are considering. 3PAR continues to lead the industry in technical innovations for thin provisioning.

Hitachi's Storage Provisioning Masquerade

The League of Suspicious Avatars had a party on Fat Tuesday in honor of Hitachi's chubby Dynamic Provisionin.  A great time was had by all and several newcomers showed up.

Terremark's Enterprise Cloud selected as the service for USA.gov

Terremark's Enterprise Cloud was just chosen as the computing service platform for the Federal government's USA.gov portal.  Read about is here. 

"The GSA will leverage the Enterprise Cloud's reliability and agility to dynamically provision highly available cloud computing resources to handle any spikes in online traffic. The on-demand nature of The Enterprise Cloud allowed Terremark to provide the GSA complete access to secure cloud-based computing resources within minutes instead of weeks. Terremark's solution will also supply GSA with industry-leading physical and logical security and robust connectivity to some of the world's leading carrier networks. "

Congratulations to everybody at Terremark for winning this business! 

3PAR is one of the strategic partners for Terremark's Enterprise Cloud. Its exciting for us to be a central part of their vision for flexible and agile enterprise computing services.

Energy savings and rebates with 3PAR's thin provisioning

Its no secret that server and storage consolidation saves customers money through energy savings.  If you are planning to make changes to your IT infrastructure make sure you include energy costs (duh!) in your plans and check out any rebates that might be available too.  For instance, PG&E in California has excellent financial incentive programs to help IT customers conserve energy - including design assistance.

Efficient storage from 3PAR with industry leading thin provisioning technology can save you a lot in energy costs.

Paycuts at HP - is this what deflation looks like?

By now everybody's heard about HP's earnings announcement yesterday.  Revenues were short by $3 Billion.  That is a mighty big miss.  I give the company's management credit for having cut expenses to compensate for the drop in sales. I'm sure it was painful and there are numerous former employees who are paying the price. None of this is good.

The problem is what to do moving forward.  If demand continues to drop and if expenses are already cut close to the bone,  it becomes very difficult to make further expense reductions without cutting products.  For a company with such a broad product set, this would be extremely hard to manage.

So, in a move that I assume will help maintain their overall business operations, HP is cutting salaries across the board, a move that will undoubtedly upset some of their workers. Some employees will probably see this as unfair, but sharing the pain is probably better than cutting employees further. I'm not picking on HP here, because I doubt this will be the last time we hear of technology companies asking their people to take pay cuts in 2009.  Pay cuts might not necessarily signal what economists call deflation, but it seems to me that when pay is reduced en-masse, that is a very string sign of deflation at work.

There are many problems with deflation - one of them is the value of debt.  While all other assets are shrinking in value, debt stays the same.  The interest might be reduced but the amount of debt does not.  A fundamental economic problem now is consumer confidence and the fact that people and companies have stopped spending.  I just bought my new Sync-Mobile (Ford Fusion) in October and am looking forward to getting a new energy and water-conserving washing machine in the near future.  But I don't think I'll be putting much on the credit cards for awhile.  

Attenda's utility computing platform continues to succeed

3PAR customer Attenda has a terrific vision for utility computing as well as the operational know-how to help their customers succeed.  BIW Technologies in the UK is the leading SaaS provider to the Architecture, Engineering and Construction (AEC)  market and has been a customer of Attenda's since 2000. They recently extended their services agreement with Attenda for another five years based on Attenda's excellent track record of service.   3PAR is very happy to be an integral part of Attend'as ongoing success.

3PAR's thin provisioning technology continues to lead the industry and is used heavily in industrial strength utility computing environments. Here is a telling quote from the article on the Attenda/BIW deal:

"With a typical 30% plus growth in client storage requirements, this platform enables BIW to scale rapidly without the need to over-provision large amounts of excess capacity."

Confusion over EFD's naming and the mid range

The term "mid-range" is intentinally vague.  In statistics, mid-range isn't all that useful.  Then there is the occasional use of mid-range in pet gifting.  Some people apparently are concerned that their mid-range education might not be good enough.   And if you google mid-range you can find surprising things that are difficult to understand.

Storage customers think of mid-range as something much more affordable than enterprise, but much more powerful than low-end.  Most mid-range storage customers are looking for versatility and ease of use because they have a lot of different hats to wear. In the storage world, vendors have had a tendency to think of mid-range as anything to sell to customers with smaller budgets. 

Then there are the market analysts.  Here is Gartner's most recent take. There is an enormous range of products in that category - what are they thinking?  It will be interesting to see how Gartner reacts to the new economic reality the next time it draws it's quadrants.

Given all the noise about how EFD's were named, it occurred to me that we are just settings ourselves up for future historical revisions.  At some point in the future EFDs probably won't be expensive any more and then are we going to have to pick the lint out of our navels again to come up with a new handle.

HP bloggers go viral with cross blog smackups

There's been a lot of smackulation over the last several months regarding flash SSDs (or EFD's, for enterprise flash doohickey). Yesterday, Alex McDonald from Netapp posted a smackup for EMC.   As EMC/Netapp smackalongs tend to do, this one generated several comments, including some from HP aimed at EMC's direction.

Storagebod and the Storage Architect also had posts recently about EFDs and Alex mentioned on Storagebod's post that Calvin Zito (HP) and Chuck Hollis (EMC) were snitting it up on his blog. Its a terrible thing when a blogger has to seek refuge on a customer's blog when there is unbridled third party smackage going on on his own blog.

It's probably a real challenge being a blogger at HP where corporate attorneys and PR folks are likely to frown on smackish talk on their morally superior blogs.  Maybe they are circumventing their internal behavior control mechanisms by Hi-smacking comment streams on other blogs?  What is this world coming to?

Smackups spread to outer space: could data be loose in orbit?

For the first time in earth's history - as far as we know - two satellites orbiting the planet crashed into each other and created a sizeable "debris cloud".  If only this had happened during the cold war, who knows how weird everything would be.

I'm sure storage people around the globe have thought about this - just because we tend to think about things that can go wrong.  A few decades ago when we started hearing about space junk, it seemed inevitable that this stuff would start making a mess at some point.

So the debris cloud could contain some amount of nonvolatile storage media with data on it.  Phone numbers, credit card numbers, passwords, photos of your local elementary school - who knows?  Of course access times to this data are infinite, but its still sort of interesting to think about anyway. Unlike media on earth, there isn't much in space to make media corrode and fall apart - so it could conceivably still be readable centuries from now.

At some point, it might percolate into the great incinerator in the sky where we can notice it for a fraction of a second. Otherwise, it will hang around up there waiting to beget further smackups.

Our data lives in growing orbits!

A suspicious interview with A-Rod about his steroid use

The following post has nothing whatsoever to do with storage, instead its about a baseball player and his use of performance enhancing drugs.  Its been a big story here in the US.

HP kills lefthand software-only option

It looks like customers with Lefthand Networks iscsi storage software running on anything other than an HP server are seeing the vanishing point getting closer and closer.  Lefthand has become an appliance component @ HP. 

Hey homey, good luck at Pano Logic

My old homeboy from EqualLogic, Bryan Cox is starting up his mojo again at Pano Logic,  the company that makes the ultra-sleek, zero client for desktop virtualization.  Bryan always shows up for the game with enthusiasm and determination.  

How wide striping eliminates hot spots and provides excellent performance

The Question:  Does wide striping lead to the development of hot spots?

I received a question on a post from last month. In a nutshell, the question posed is: How come multiple applications (database applications in particular) don't create disk contention problems (hot spots) for wide striping arrays?  I'm going to do my best in this post to explain how it works.

To elaborate on the question, hot spots in an array occur when an individual disk drive is overloaded with I/Os and does not have the ability to respond in the expected or desired time. The term originated with database I/O performance analysis, where a relatively small segment of database data (the hot spot) is responsible for a high number of disk reads and writes that overload a single disk drive. In a shared storage array, the occurrences of hot spots can be compounded by having two or more applications generate I/Os localized on a single disk drive, overloading it and degrading the performance of multiple applications.  Given this possibility, why would a customer purchase an array that might exhibit these problems?

The answer:  Wide striping with chunklets randomizes I/Os, minimizes seek times and avoids hot spots

Wide striping is a disk layout approach that uses a large number of disk drives (or spindles) to place data in an array. It's well recognized that putting more disk spindles to work is the way to increase an array's I/O performance. Wide striping in a 3PAR InServ array spreads the data from multiple applications over a large number of disk spindles, which simultaneously provides both high I/O levels and high capacity utilization. The spindle count obviously matters a great deal because each drive provides incremental I/O processing power. FWIW, 3PAR InServ arrays typically have hundreds of drives, but can start with 50 or so drives in a smaller configuration. 

I/Os in a wide striping array are an example of a stochastic process where I/Os are generated with varying degrees of randomness. At any given time, there are a certain number of I/Os pending for the array, which are concurrently being serviced by a large number of disk drives. It also means that the I/Os for most applications, including databases, move from one spindle to another in a random fashion, which is why this technique is referred to as randomizing I/Os.  As the I/Os for an application change spindles, individual disk drives are not overloaded and hot spots are avoided. 3PAR InServ arrays also have cache memory that provides additional performance acceleration and reduces disk contention.

Equally important, but far less obvious is how data is laid out to support a mixed application workload.  Data layouts that exacerbate disk latencies should be avoided, such as long seek times that occur when successive I/Os must jump from one disk partition to another. Storage allocations and data layouts in 3PAR systems use small disk segments called "chunklets".   Chunklets are placed in close physical proximity to each other, which means that seek times for I/Os from multiple applications are minimized.  A human being could try to figure out how to layout data for a wide striping array and never be as good as the algorithms in a wide-striping array controller.

Is there a way to measure or predict the effectiveness of wide striping in eliminating hot spots?

Just as the metric IOPS indicates how much work a disk drive can perform, we could use another metric for the ratio of spindles available to service a given I/O load. This should help us understand the ability of the array, LUN or RAID set  to handle hot spots.  Since hot spots result from disk I/O operations, which take place in approximately one centisecond (cs, hundredth of a second) I created this little hot spot metric: spindles per IOs generated each cs, or spindles/IOP(cs). 

A spindles/IOP(cs) ratio greater than one indicates there are probably sufficient spindles to do the work without developing hot spots; and a ratio less than one indicates that hot spots could develop that would degrade performance. Of course, the interpretation of this ratio should change depending on the specs of the drives under consideration and any other elements that impact the average service time of the drives.  FWIW, the radial interleaving of chunklets is an example of an array technology that would reduce the average service times of drives in a wide striping array.

I thought I'd suggest a couple numerical models to try out my hot spot metric.  Let's say there is a hundred-drive wide striping array with an load of 100 I/Os generated during any centisecond interval. This corresponds to a spindles/IOP(cs) ratio of 1:1.  The statistical odds are good that many of these pending I/Os will be serviced by different spindles - and when there is contention for a spindle, it's unlikely that the same applications will continue to generate subsequent I/Os for the same spindle due to I/O randomization.

In contrast, consider a traditional narrowly striped array that has ten drives servicing a transactional database application. The average number of I/Os generated by this application every centisecond is fifteen, which means the spindles/IOP(cs) ratio is 0.67 (10/15).  If one looks at the aggregate maximum IOPS of the drives, they could come to the conclusion that these ten drives have plenty of performance overhead to handle the workload, however, the spindles/IOP(cs) ratio indicates that hot spots could develop. Certainly some people reading this will be able to relate to the experience of configuring RAID sets that should have adequate IOPS for a given workload - only to find that hot spots have developed, creating infuriating application performance problems. 

So what of it?

The numbers used in the examples above were pulled out of thin air based on estimates that seemed reasonable to me. 

One of the more difficult aspects of trying to use the hot spot metric is figuring out how many I/Os your applications are generating.

I'm interested in hearing what others think of the hot spot metric.  Usual blogging rules apply - which means it's open to the mosh pit.