A lot has been written about HP's acquisition of 3PAR. I see it as a real game changer in our industry. This screencast explains why.
A lot has been written about HP's acquisition of 3PAR. I see it as a real game changer in our industry. This screencast explains why.
Posted at 09:25 PM in 3PAR, Adaptive Optimization, Autonomic, Efficient, energy, enterprise storage, green computing, HP, multi-tenant storage, performance, reservationless, storage management, storage services, thin provisioning, tiering, utility computing, video, virtualization | Permalink | Comments (2) | TrackBack (0)
Tags: 3PAR, Autonomic, Converged Infrastructure, Efficient, HP, Multi-tenant
3PAR designs its systems to provide huge time savings for storage administrators. Below is a video of our new InForm Management Console (IMC) 4.1, announced today, showing how incredibly easy it is to configure and operate 3PAR's Remote Copy application.
Things that the demo didn't show that are advantages of 3PAR's single software architecture are:
Here is a brief description of all the software functions available through IMC 4.1. As you can see, it's a pretty comprehensive list of features:
Posted at 06:00 AM in 3PAR, Adaptive Optimization, clustered storage, Dynamic Optimization, enterprise storage, mid range storage, multi-tenant storage, remote copy, snapshots, storage management, thin provisioning, tool talk, utility computing, video, Virtual Domains, wide striping | Permalink | Comments (3) | TrackBack (0)
Tags: 3PAR, console, GUI, IMC 4.1, remote copy, replication
It's getting very hard to keep up with all the crazy social media stunts coming out of Hopkington, but they seem to have done it to themselves again. First was the questionable spamming for viewers so they could claim they had a viral video, then today they just "leaked" a 3PAR sales "kill sheet" - and also apparently established a "secret" site with the URL Notapp.com, where they compared their own guarantee program to Netapp's. According to Simon Sharwood at Search Storage Australia, the site was removed and accessing the URL directed browsers to EMC's site.
Perhaps it is all part of a new marketing strategy by newcomer Jeremy Burton, who joined EMC as Chief Marketing Officer back in March. As best I can tell, Burton's new marketing strategy for the company is that people will believe anything. Maybe he doesn't think there are enough new products coming out of EMC - or that the delays in getting their ballyhooed FAST out the door are too embarrassing - but instead of trying to promote EMC on its own merits, it looks like he is doing his utmost to mud wrestle. Is that what EMC is paying him the big bucks for?
EMC suddenly is taking a bigger interest in 3PAR. That's good. Search Storage Australia just published parts of a competitive document that EMC was circulating to it's partners about 3PAR. It certainly wasn't a surprise because we'd seen it previously, but I was sorry to see it published because it made EMC look ridiculous, which was working pretty well for us. But now that it's been outed, here is what we have to say about it (in the guise of Ineption's lead character, the CRO)
The messaging is not built in, but our zero detection technology for optimizing capacity is. The host SW commands to do this are short and do not require "careful coordination". Veritas, Oracle, Windows Server and Linux software all work with minimal operator effort. For instance, this document from Oracle, describes the whole process, with the sole operator command being this: #bash ASRU LDATA.
Can EMC provide online reclamation of zeroed space without risking capacity overruns and with tolerable performance? 3PAR can. Does EMC have these capabilities in both mid-range and enterprise storage arrays? 3PAR does.
3PAR has both Flash and 1 TB SATA drives. We also have Adaptive Optimization software that uses Flash SSDs for storage tiering. EMC still doesn't have it after they made such a big deal about it last year. They like to tell customers that their size gives them development advantages, but their track record doesn't support their claim.
3PAR arrays allow users to create many tiers, but without the need for disk pools. Tiers are constructed from the combination of drive type plus RAID level. For instance, you can have separate tiers for SATA, FC and Flash SSD drives with the RAID level you select. Our Dynamic Optimization software allows admins to move data from one tier to another. You can "dial in" the performance and protection you want.
All systems have a peak output , ours just happens to have a lot more throughput than theirs - and at higher disk utilizations. We have published benchmarks that show how our systems perform. They don't. Adding disk drives to a system and utilizing those drives is far easier with a 3PAR system than either VMAX or Clariion where you have to wrestle with putting drives in the pools you want to use them for.
There are no disk pools in 3PAR storage. Pools trap resources so you can't use them. Work isolation in pools leads to hot spots and storage admin nightmares. Wide striping does not mean you can't have tiers. That is an idiotic statement.
VMAX can configure large pools - and all the drives in them have to be at the same RAID level meaning you can't create multiple tiers within those pools. If you want multiple tiers, you need multiple pools and all the headaches that involves. Change management in an environment with multiple pools is complicated. You also need to consider the pools needed for snapshots and remote replication. Are those easy to provision and change on EMC storage. Most would say "no".
3PAR uses all disk spindles all the time for delivering IOPS and pro-active sparing is done using reserved space on those drives. Rebuilds do process quickly. Would EMC have you believe they never have to perform drive rebuilds? Really?
Our front end archiecture was designed for large-scale parallel connectivity to match the massive bandwidth capabilities of our wide striped back end. Our benchmarks and the cost per IOPs in those benchmarks speak for themselves. Our customers also tend to run 3PAR systems at much higher disk utilizations than they run other vendor's arrays.
We support a huge number of ports on our systems w/full active/active data access across all controllers. All controller nodes can be used to access all data volumes. We have a number of customers that run fairly sizable SANS without switches because they have enough ports on their arrays so they don't need to consolidate access through switches.
5- 9s? We're there. Our systems get pounded on every day in some of the largest private and public data centers in the world. They are designed with complete redundancy in all components and have advanced capabilities such as Persistent Cache to maintain high levels of performance even after the loss of a controller.
The delays in bringing their FAST tiering software - a product they were hyping in April of 2009 - to market have shown that size doesn't matter much when it comes to delivering technology on time. I'm not saying 3PAR always delivers on time, but EMC is far from immune to these problems. In fact, the need for them to coordinate across multiple product lines creates certain disadvantages for them.
As to their comments on our support; they are pure FUD and grasping for straws. We would not be able to maintain the customers we have if it were not for our efforts at supporting them.
* * * * * *
The following content was added on July 30th by Rusty Walther, 3PAR's Vice President of Customer Services & Support.
Stating that 3PAR “outsources support” is just plain silly, especially coming from a company that keeps most of the worlds’ largest offshore outsourcing companies in business. Like EMC, 3PAR uses Third Party Maintenance suppliers (TPM’s) for break-fix field activities. In some geographies, EMC and 3PAR even use the “same” TPM. But EMC also outsources most of their volume call center
and Level-1 Technical Support to offshore suppliers. Not so at 3PAR. Everyone that touches a 3PAR support case is a 3PAR-badged employee. I challenge EMC to identify a single outsourcing company that handles 3PAR technical support. EMC’s outsourced technical support sub-contractors could be listed alphabetically, by geography, or by technology category … but you’d need a couple of sheets of paper to do it.
Posted at 03:40 AM in 3PAR, Adaptive Optimization, benchmarks, bloggers, clustered storage, customers, Dynamic Optimization, EMC, enterprise storage, flash, INEPTION, mid range storage, multi-tenant storage, Oracle, performance, remote copy, SAN, SSD, storage companies, storage management, thin provisioning, tiering, wide striping | Permalink | Comments (6) | TrackBack (0)
Tags: 3PAR, array, EMC, FUD, Netapp, Netopp, Oracle, RAID, storage, tiering, VMAX
1. Everybody has it.
2. All implementations are the same.
3. Compellent has the most full-featured implementation.
4. It will use expensive SSD resources for low-priority apps.
5. It won't work for VMFS data.
6. EMC is shipping it.
7. Tiering+SATA+SSDs will eliminate FC/SAS drives.
8. It only works with SSDs.
9. You will end up thrashing data across tiers.
10. You have to buy it before you can determine if it will work for you.
3PAR's sub-volume tiering, Adaptive Optimization (AO), comes with a lot of features that make it much better than what competitors say. It doesn't require SSDs and it's not for everybody, but if you need it, it will do the job.
Dave Vellante from Wikibon posted his summary of 3PAR's analyst call this week. It's a good chance for people to get an unvarnished opinion how 3PAR is doing.
One of the things that came up during the call was a discussion of Storage Federation. Here is what our CEO David Scott said were the three main points to understand about it:
Storage Federation is a new concept that has been getting a lot of attention from storage bloggers all of a sudden. People interested in following these discussions can start on Stuiesav's blog and follow the links in his posting.
A good analogy for Storage Federation is that storage systems work together in a team. Each member of the team has the ability to manage it's own resources and can make the information about its resources available to the team so that they can act together according to internal algorithms and policies.
Autonomic management is scoped from the individual storage system to the Storage Federation. The same types of algorithms and policies that relocate volumes within an individual array are used to relocate volumes within the Federation. For instance, the autonomic management technology used in 3PAR's volume and sub-volume relocation software, Dynamic Optimization and Adaptive Optimization, could be adapted for use in 3PAR's Storage Federation.
The graphic below illustrates the two levels of autonomic storage management. On the bottom, three storage systems manage their own resources and share resource information to the Federation where they can be accessed and managed by Federal algorithms and policies. If you click the graphic, you will see a slightly larger version that is easier to see.
Some of the thorniest storage ownership problems can be addressed by Storage Federation, including load balancing, firmware upgrades, system maintenance and data migrations.
Posted at 01:16 PM in 3PAR, Adaptive Optimization, bloggers, Dynamic Optimization, enterprise storage, Federated Storage, mid range storage, multi-tenant storage, storage management, tiering, unified computing | Permalink | Comments (0) | TrackBack (0)
Tags: 3PAR, Autonomic, federated, federation, optimization, policies, storage
I wrote a post yesterday that showed IOPS calculations for a few different native wide striping configurations and I thought I'd add storage tiering to the mix today. Native wide striping places data from all volumes across all drives in the array (or of a certain drive class if you have mixed drives in your array) and randomizes workloads across all resources. The biggest advantages of native wide striping over traditional array designs that rely on multiple pools and workload isolation are:
Although native wide striping can handle complex, mixed workloads of transaction and sequential data access, applications that are either latency sensitive or single threaded can significantly increase their storage performance through the use of SSDs and storage tiering.
3PAR's software for storage tiering is called Adaptive Optimization, or AO. Based on administrator policies and algorithms keyed off QOS gradients, a 3PAR InServ array autonomically copies data from lower IOPS disk drives onto high IOPS SSDs.
The 3PAR tiering solution uses STEC MACHIOPS SSDs with a sustainable I/O rate of 10,000 IOPS. These devices have 50GB capacities and are installed as sets of eight SSDs across all mesh active controllers in a 3PAR InServ array to balance the high IOPS workload load over all controllers as well as drives.
Below are a few calculations for maximum sustainable IOPS from InServ arrays that use both SATA drives as well as SSDs with AO. I used 5,000 IOPS as the metric for calculating SSD performance, which is a conservative estimate for the STEC MACHIOPS performance, but actual performance from an AO-enabled array could be lower due to a number of variables including the I/O activity levels that can be sustained by both applications and servers, policy settings made by storage administrators, the accuracy of algorithms to select data for tiering and copy operations that populate and de-populate the SSDs.
Storage tiering is still in it's early stages and the industry is going to learn a great deal about this technology over the next several years. Performance models will certainly evolve as key variables are identified, which will almost certainly include server and application components.
Array 1: 160 SATA disk drives; 80% reads, no SSDs
Total IOPS of all drives in the array: 12,800
IOPS delivered to all servers w/RAID 5: 8,000
IOPS delivered to all servers w/RAID 10: 10,667
IOPS delivered to all servers w/RAID 6: 5,998
Array 2: 160 SATA disk drives; 80% reads - 8 SSDs; 50% reads
Total IOPS of all drives in the array: 52,800
IOPS delivered to all servers w/RAID6 (SATA) & RAID5 (FC): 21,998
Array 3: 160 SATA disk drives; 80% reads - 32 SSDs; 50% reads
Total IOPS of all drives in the array: 172,800
IOPS delivered to all servers w/RAID6 (SATA) & RAID5 (FC): 69,998
Array 4: 480 SATA disk drives; 80% reads - 32 SSDs; 50% reads
Total IOPS of all drives in the array: 198,400
IOPS delivered to all servers w/RAID6 (SATA) & RAID5 (FC): 81,994
Even a relatively small amount of SSD storage can boost performance approximately four times, as shown by Arrays # 1 and 2 above where eight SSDs totaling 400GB were added to an all-SATA configuration. It's also interesting to note the performance differences between arrays #3 and #4 above. Although the number of SATA drives tripled, the IOPS performance increased only 15%.
Latency-sensitive applications are the best candidates for storage tiering to SSDs with 3PAR's AO (Adaptive Optimization.Typically, these are:
People ask about Microsoft Exchange and I tell them it benefits a great deal from big, wide striping, but not much from tiering because Exchange performance is mostly a matter of providing adequate throughput.
An app that people run daily but is seldom associated with transaction processing is backup. This SWCSA video discusses backup as well as the prevailing shift to dashcams and the implications for SWCSA branding.
Chuck Hollis had an excellent post last week, discussing caching.
About 10 years ago a small team that I was a part of looked at starting a company that would do something similar to what IBM's SVC does. The idea was to create a SAN front end controller with a lot of cache memory that would virtualize "downstream" storage and provide performance boosts through various techniques such as caching, striping, and multi-way mirroring. We gave up on the idea when it became apparent to us that the project was quite a bit larger than we initially thought and it was unclear when we would ever have sufficient resources to get a competitive product to market. I think we could have sold the idea to venture capital investors who were throwing money at storage startups, but we couldn't sell it to ourselves. For those of you that wonder why I tend to think SVC is an important product, that's why - I know some of the things IBM did to make it work and I admire their ability to bring it to market.
Anyway, one of the hurdles we couldn't get past was how to deal with mixed workloads originating from multiple SAN attached servers simultaneously streaming data from data warehouses and IOPS from transaction databases and all sorts of other bursty, unpredictable applications competing for memory and disk resources. As a front-end appliance, you can control your own cache resources, but you can't do much about the back end disks because downstream arrays mask them from the appliance. In fact, there was no way to predict the performance of the downstream arrays for any given workload. We consulted with experts from the industry and research universities and they were all discouraging about the ability to significantly improve the performance of mixed workloads in a shared SAN appliance.
There were two issues: downstream arrays already had their own caches and the dynamics of sharing cache resources in an appliance. If these fundamental problems could be solved, the rest of the work was the not so simple matter of making it work as a cluster with mirrored cache for HA.
Uncoordinated multi-level caches have the problem of redundant data. For example, pre-fetching data in the appliance's cache will likely end up loading the same data in the downstream array's cache. With duplicated cache data, a cache hit on the appliance won't be that much faster than a cache hit in the downstream array - and a cache miss in both will be much slower. It's difficult to prove the value of THAT. Its certainly possible to tune both caches differently, but this turns out to be easier said than done and tends to flatten the value of an "easy to use" appliance.
So, the way to overcome this is to make a HUGE cache in an appliance. Increasing a cache's size can do a lot for performance and Chuck said,
The exception, of course, is if you've got the bucks to create a ginormous read cache, and pull almost all the significant data into memory. Don't snicker -- there are a few use cases where this sort of approach makes sense.
In other words, create a RAM disk in cache. This technique is normally reserved for a single, high profile application and as Chuck wrote, there are use cases where this makes sense. But it doesn't address the requirements of mixed workloads where there are a large number of applications that do not merit dedicated memory, but still need good performance. It might be possible to micro-manage cache for some number of applications by dedicating cache for each of them, but that requires a great deal of work that is likely going to be a temporary solution lasting a couple of months at most. It's probably a great way to drive storage admins crazy.
A more palatable approach is to use a global cache that shares cache resources among multiple servers and their applications. In some cases, it's possible to predict workload demands (end of month processing for example), but in many cases, the instantaneous performance requirements cannot be predicted because it is driven by spontaneous events. As many people are too familiar with, spikes in Internet activity and the corresponding bottlenecks in back end storage clearly elaborate the challenges of mixed workloads. Global caches that can accommodate large Internet traffic spikes are expensive and do not provide noticeable performance advantages most of the time - they are overkill.
The proliferation of virtualized servers has significantly increased the breadth of the mixture that a storage array has to deal with. In general, there is more overall I/O activity that is, for the most part, less predictable, and therefore more difficult to deal with in cache.
The question is, is storage tiering with SSDs any better? Possibly. If it is going to more effective than caching, it needs to be able to provide more control points and intelligence than cache typically does. For instance, the ability to prioritize and schedule applications for movement into SSD tiers could be an important difference. 3PAR's QoS Gradient concept in Adaptive Optimization is an example of a simple prioritization scheme. The internal counterpart to QoS Gradients is internal monitoring of I/O levels which help determine which applications are promoted.
That's not to say caching can't have some of the same controls, but traditionally caching has been more reactive than proactive. To be clear, tiering is also reactive, but within the context of intelligent preparation and business-driven policies.
Still, considering the cost of SSDs, you have to ask the question is tiering overkill too? It might be. If the array can keep up most of the time, how much SSD capacity should be purchased? These are the sorts of things that will be determined over the next couple years.
The best technology to date for dealing with mixed workloads continues to be wide striping. If you throw out latency-sensitive applications from the mix - those are the same apps Chuck Hollis referred to when he talked about the applicability of "ginormous read caches" - then the array just has to provide adequate throughput at reasonable service times (latencies).
Wide striping does this by spreading the workload mix over as many disk drives as possible. Not the number of drives that fit in a shelf or can be added to a RAID group, but all the drives in an array, at best or all the drives of a particular class, day SATA or FC. Wide striping very thin layers of data across hundreds of drives means that hundreds of servers can be accessing data simultaneously, with a minimal amount of contention. The result is that all the drives are kept busy at the same rate and that none of them bear an unfair burden. The overall sustainable throughput is very high, scales by adding more drives and in general fits the profile of mixed workloads better than any affordable caching scheme does.
We've become accustomed to thinking that more memory is the answer to all storage performance problems, but that doesn't exhaust the potential of massed disk drives.
Posted at 01:26 PM in 3PAR, Adaptive Optimization, bloggers, cloud computing, customers, EMC, enterprise storage, IBM, mid range storage, performance, servers, SSD, storage management, tiering, virtualization, wide striping | Permalink | Comments (0) | TrackBack (0)
Tags: 3PAR, cache, EMC, Gradient, mixed workload, QoS, SSD, storage, tiering, wide striping
Separated at birth?
There have been some interesting discussions lately about storage tiering And just because 3PAR beat most everybody else to the punch this week with our AO announcement, I think it's important to keep things in perspective - storage tiering does not solve everybody's problems. Here is my top 10 list of reasons some people don't want to embrace tiering:
10. Chuck Norris does not need automation, automation needs Chuck Norris.
9. The Chuck Norris of IT, Larry Ellison, says Exadata is the only tier you need.
8. Chuck Norris & Larry Ellison will race to determine if tiering is good or evil.
7. Heard it uses gerbils, which is unfair to guinea pigs.
6. Chuck Norris got to Tom Georgens, he will get to you too.
5. Chuck Norris designed 3PAR's wide striping - who needs tiering?
4. Chuck Norris already tunes my storage in his sleep.
3. Larry Ellison is going to buy all the world's SSDs and corner the market.
2. Waiting to hear if Chuck Norris will discuss tiering this week on InfoSmack.
1. Too busy celebrating Chuck Norris' 70th b-day today to care about tiering.
Yesterday, 3PAR announced Adaptive Optimization (AO), our solution for storage tiering and support for SSD flash drives. Here are the elements of this technology that I believe will have the most impact on customers and the rest of the industry.
1) Tiering works by making copies of data on lower cost, low-IOPS storage to high-IOPS storage - and back again. Storage tiering has been associated with ILM, which assumed data is initially located on more expensive, high-IOPS storage and, as it ages and is accessed less frequently, is moved to lower-cost, low-IOPS storage. The perception that tiering implies fast to slow data migration was reinforced by Compellent with it's early entrant storage tiering technology, Data Progression.
The economic benefits of tiering are much more compelling if data is originally located on low-IOPS storage and then moved to high-IOPS storage when it becomes useful to do so. This reduces the amount of high-IOPS storage that needs to be purchased and reserves high-IOPS storage for the applications that need it the most. This model of promoting data to high-IOPS storage will replace the old model of data "trickling downhill to cheap storage."
2) Sub-volume tiering means high-IOPS storage can be reserved for high-IOPS work and effectively shared by the applications that need it the most. AO copies data in 128 MB sub-volume regions that contain specific RAIDed volume slices. Many physical and virtual servers can have their volume's most active regions located in high-IOPS storage capacity at the same time.
Data redundancy is accomplished when AO reads data from it's source region and restripes it into a region on the target tier - using the RAID level of the target. AO allows data to be protected by whatever RAID is appropriate for the tier and the data. 3PAR's chunklet architecture is maintained for SSDs, which means a SSDs in an InServ array can apply several different RAID levels simultaneously. Every vendor's sub-volume tiering technology will be different, including the number of ways devices can be combined in RAID and how wide striping can be applied.
3) Tiering does not mean you have to buy SSDs to make it pay off. Tiering is a cost-reduction technology. One of the most obvious ways to reduce the cost of storage is to buy cheaper disks with higher capacity, such as SATA drives.
The regions used by AO are the same on-disk structures that 3PAR uses for it's Dynamic Optimization (DO) software that re-levels volumes across disk drives in an InServ array. A customer with all FC drives in an InServ array could take advantage of both AO and DO by increasing the capacity of an array with SATA drives, using Dynamic Optimization to redistribute their volumes across the SATA drives and then using FC drives as their high-IOPS AO tier. This way, they can continue to get the IO rates they expect, but reduce the cost of incremental capacity as they upgrade their system.
4) The system determines what to move and how to move it. I/O density rate is a term that refers to how much data access occurs in a region over a given amount of time. AO recognizes region candidates for tiering by their I/O density rates.
Administrators control the AO participation for each volume by assigning them to an AO Profile and a QoS Gradient. The profile is a short stack of device-RAID levels, such as SATA RAID 6, FC RAID 5(7+1) and SSD RAID5 (3+1). AO allows either 2 or 3 device RAID levels in the profile's stack.
The QoS gradient is a relative determinant of how quickly the volume will be acted upon. I like to think of it as something like different viscosities for different fluids, but for storage. AO today has three QoS gradients, performance, cost and balanced.
Back in Novemebr, Tony Asaro wrote about his discussions with HDS' storage customers regarding storage tiering.
Another discussion was around using policies to automate the process. One group was a bit concerned about automating this process but realized that, again, with PBs of data being stored that the only way to effectively implement intelligent tiered storage is via automation. Additionally, it is not an all or nothing proposition. You can select certain volumes and applications to implement and gain a comfort level before deploying more widely. One of the key tenants of technology is to automate otherwise manually cumbersome processes. We just need to get over that hurdle but we need to do so in a planned, considered and reasoned way.
By applying measured I/O density rates, AO profiles and QoS Gradients, 3PAR has taken the first major steps to automating storage tiering and removing the burden from administrators.
5) Tiering can and should scale out. David Floyer from Wikibon wrote a good piece yesterday on our announcement where, among many things, he discussed how 3PAR is using smaller SSDs spread over more controllers:
....it spreads a small amount of SSD amongst the 3PAR engines so the IO’s aren’t all going to a single drive and sucking up a lot of bandwidth – it’s nicely balanced. Traditional implementations will use larger drive with more IO’s going to that drive. The part of the array with that drive will get more activity.
In practice we don’t think this will matter all that much because, for example, EMC’s V-Max has more bandwidth to play with than 3PAR and EMC uses its cache to transfer data between tiers to avoid bottlenecks. Nonetheless, on paper, the 3PAR implementation looks to be more efficient which means (in theory) it can do more with less flash. But nobody really knows yet.
3PAR storage arrays avoid I/O bottlenecks by incorporating tiny virtual storage elements (chunklets) and spreading the workload over as many devices and controllers as possible. This approach differs from other vendors where smaller groups of resources are created and then combined into larger constructs that are more cumbersome to manage and tune than a single widely distributed storage span. The same concepts apply to SSD integration, where InServ arrays accommodate multiples of many, smaller sized SSDs for scaling out high-IOPS tiers for those customers that may want to expand their use of AO in the future .
Posted at 12:15 PM in 3PAR, Adaptive Optimization, bloggers, cloud computing, clustered storage, Compellent, Dynamic Optimization, enterprise storage, flash, HDS, performance, SSD, storage management, tiering, utility computing, wide striping | Permalink | Comments (3) | TrackBack (0)
Tags: 3PAR, Adaptive, AO, Compellent, EMC, EMC, optimization, QoS gradient, SSD, storage, sub-volume, tiering