Running detailed research since 2002 allows us to define recurring industry patterns and trends newcomers miss, and also to reminisce a little. This looks back at our prior storage research to better understand some of the aspects of today’s technology. (The original 2002 report is outside our paywall here.)
Sometimes, you just have to laugh. In 2002, we considered ‘Over 50TB’ as the highest category needed for SAN capacity! We were right – the energy/utilities vertical had the highest count, with only 40% of respondents falling into this category. For industry veterans, perhaps you recall some of these companies mentioned that have since disappeared or now only exist in stock codes, including Compaq, Legato, Veritas, StoreAge, BlueArc or StorageTek. If you have a tchotchke with the brand of one of the departed companies, tweet a photo with a #TIPflashback tag so we can see it too.
The real point in looking back is to acknowledge the issues we solved, and spot those that may recur. Consider the chart below.
Fourteen years ago, managing growth was still the leading pain point in storage, and familiar ones around scalability and migration were only just arriving. But in 2002, we were still taking the islands of DAS storage capacity attached to specific compute environments, and merging them into SAN and NAS sharable environments. See the 4% selecting ‘internal selling of the SAN concept,’ the 5% ‘developing storage networking strategy’ and the 12% wanting to ‘consolidate existing capacity.’ We were well aware of the capacity lying fallow in silos across the enterprise, and the amount of duplicated technology appearing in those silos. We were determined to merge those ‘islands’ into a capacity ‘continent.’ As an industry, we succeeded.
These sorts of pain points could return if storage is included in converged infrastructure procurement. Certified environments to help avoid interoperability issues make a lot of sense. But do we want to buy storage capacity at the same time, and in proportion to compute capacity? It is hard enough to predict future capacity needs. The idea of purchasing compute and storage capacity in equal proportions assumes we can successfully calculate a compute-capacity ratio for an application. Getting the ratio wrong means we end up with islands of free capacity in the wrong converged infrastructure racks … again.
Perhaps the potential waste doesn’t really matter. The cost of wasted capacity is not what it was in 2002. Few businesses would like to admit over-purchasing to need, but this is likely less expensive than reacting to exhausted capacity. Reactive additions of capacity are not as painful as they were then, as newer storage architectures permit disruption-free addition of capacity. There will still be waste when compute or storage grows faster than the other requiring paired purchases when only one is really needed. (Did anyone have to buy a second Vblock when they really only needed a little more speed?) The connectivity and interoperability challenges between converged infrastructure stacks are also not what they were. Networking has standardized, and even when physical connectivity varies (e.g., InfiniBand), today they support standard protocols further up the stack, allowing interoperation.
Still, we spent a lot of time and effort since 2002 corralling those silos of proprietary storage into centrally managed storage plants. We know how complicated that task was. Storage pros will find it hard to let go of the concept for a converged infrastructure model that includes storage capacity. Does it make sense to include storage in converged? Convergence makes a lot of sense in transient parts of the stack like compute and network, but the permanent nature of storage is a significant difference. Try calculating the compute-capacity ratio for one of your longer-life applications and see if they trended equally over time.
Yes, we have been keeping narratives to provide context around the data since 2002. Here are a few that capture that moment in time. Note that some ideas like ‘storage on demand’ have been around for a while.
- “We have so many islands of SAN that we are looking at Director switches to combine them. We would like to meld the block level SAN access with NAS file access all behind a Fibre fabric.” – a 2002 Fortune 500 communications products company
- “Our biggest issues are: We have so much DAS that we can’t provision where it’s needed accurately. Right now, we have islands of capacity until we get into SAN or NAS environment. We have difficulty getting vendor support in moving to a storage-on-demand model. We have no good mechanism to understand what we should charge back to users.” – a 2002 F100 consumer products company
- “We are now consolidating on smaller group of servers, but we did a big expansion recently in the data warehouse area. We will wind up with less data overall. In North America alone, we have 450 NT servers. We should be able to get that down to 200 servers. Some servers are running at less than 1% of capacity. This happened because we weren’t monitoring servers. But now we are, as part of our overall architecture revamping. We’re now examining our cluster options and moving almost all of our capacity to SANs.” – a 2002 F500 chemical company
- “Our biggest issue is integrating islands of DAS into a switched storage network. Also, we are trying to get economies of scale and improve our utilization rates.” – a 2002 F500 communications services company