Busted! Debunking 7 Enterprise Data Storage Myths About Software-Defined Storage
by Michael Fiorenza on October 15, 2016
The amount of data being generated, consumed, and stored by companies around the world is growing at an exponential rate. Software-Defined Storage (SDS) is being hailed as a revolutionary new technology for managing this unruly glut of information in a way that's less costly, more flexible, and more reliable than traditional storage solutions.
Yet, although the use of SDS is accelerating, its adoption is being hindered by uncertainty about what it is, and what it can and cannot do. In this article, we'll try to clear up some of the confusion surrounding how SDS should be defined, and whether it should be considered a viable option to meet the storage challenges companies are facing today.
What is Software Defined Storage?
The term "Software-Defined Storage" denotes a system in which software manages the data store independent of the physical devices used to provide that storage. Systems using SDS deliver their data requests exclusively to this intervening software layer without having any awareness of the nature of the underlying physical storage.
This decoupling of the control interface from the hardware and firmware used to store and retrieve the data is the basis for SDS features such as theoretically limitless scalability, the ability to use inexpensive commodity hardware and storage media from diverse manufacturers and to mix them as desired, a high level of reliability and fault tolerance, unified storage management, and geographically unrestricted backup and disaster recovery.
Now that we know what SDS is, let's deal with some of the myths that have arisen about its capabilities.
Myth #1: SDS Is Just Vendor Hype
As with any hot new technology, in the early days of SDS a number of sales hungry storage vendors repackaged existing products and put the Software-Defined Storage tag on them. Even the Storage & Network Industry Association (SNIA), in an otherwise approving 2014 report, noted that the term SDS was being used as a "marketing buzzword."
However, as information technology research firm 451 Research observes in their 2016 report on storage trends, "The concept of software-defined storage (SDS) has moved past the marketing hype phase."
Certainly, many storage manufacturers and their customers think so. According to Scott Sinclair, a senior analyst with the Enterprise Strategy Group, about 68 percent of the organizations they surveyed have adopted SDS as a long-term strategy. And storage behemoth IBM recently announced that it is investing $1 billion in SDS.
An article published by PRNewswire reports that "The total Software Defined Storage market is expected to grow from $1409.7 million in 2014 to $6217.6 million by 2019, at an estimated Compound Annual Growth Rate (CAGR) of 34.6% from 2014 to 2019."
Although some sellers have misused the term SDS in an attempt to quickly get a foothold in an emerging market, software defined storage is real.
Myth #2: SDS Is Just Another Name For Virtualization
Virtualization, which has a long history in the storage arena, is the basic concept underlying SDS. Perhaps the most widely used example of virtualization in the storage industry today is the Storage Area Network (SAN). With SAN, virtualization is used to provide an abstraction layer between storage hardware and the using system, so that attached clients can address a SAN network as if it were a disk drive. SAN can pool the resources of a number of storage arrays from multiple vendors behind its software interface.
SDS, however, provides an additional software layer that can manage a practically unlimited number of SANs using policy-based automation for resource provisioning. SDS also allows programmatic control of storage through APIs. The SNIA describes SDS as "Virtualized storage with a service management interface." The unique value of SDS is embodied in the highly sophisticated software of that service management interface.
Myth #3: SDS Is Too Complicated
Eric Carter, Senior Director of Marketing at Hedvig, has observed that "Storage used to be a domain exclusively reserved for highly-trained specialists." Because of the complexity of the operations required to manage the technologically and geographically diverse resources of a SDS implementation, it may appear on the surface that Carter's adage applies even more strongly to SDS.
Yet, according to an article in ComputerWorld, "The goal [of SDS] is to make it easier for administrators to flexibly manage a variety of storage devices via software and automated policies." The article goes on to note that "enterprise IT managers give a high priority to simplicity and ease of use." And that's exactly what sophisticated SDS providers, such as EMC, VMWare, IBM, and Cisco, have designed their products to supply.
A defining concept for SDS is that it provides a common, consistent, easy-to-use interface for managing an enterprise's storage, whatever the mix of underlying physical resources.
Myth #4: SDS Is Only For Large Enterprises
An outgrowth of the "too complicated" myth is the idea that SDS is only suitable for large enterprises that are able to devote teams of sophisticated IT specialists to the care and feeding of the organization's storage infrastructure. When it first came on the scene, SDS was not so much a "product" as a concept - it was not something you could buy in a box. So some larger companies led the way in adopting the technology by adding their own discrete storage units to specialized management software and melding it all into a unified SDS system. The level of expertise required to put together and manage such implementations was indeed high.
However, the days when the only way to assemble a SDS-based storage system was to do it yourself have passed. Now, SDS solutions can be purchased as hardware/software bundles in pretty much the same way storage units have traditionally been acquired. And, as noted above, the management software is specifically designed for ease of use by less technically sophisticated personnel. With these innovations, smaller organizations are no longer foreclosed from participating in the SDS revolution by the necessity of employing expert IT staff to assemble and run the system.
Myth #5: SDS Is Only For Smaller Companies
The inverse of the "for large companies only" idea is the myth that only SMBs/SMEs need SDS. When SDS was first introduced, many assumed that larger enterprises that had a well-entrenched staff of storage and admin experts would derive little added benefit by moving to SDS. It was thought that small and medium-sized companies, with their far greater budget and staff limitations, would be helped the most by the operational and cost benefits SDS provided.
But now, with the unparalleled scale up and scale out capabilities it offers, along with the flexibility to seamlessly mix a company's legacy storage systems with newly acquired resources and manage the entire system as a unified whole, SDS has become the storage technology of choice for organizations of all sizes.
Myth #6: SDS Is Too Slow
One of the great advantages of SDS is that it allows use of inexpensive commodity hardware in its storage arrays. The most cost-effective high-capacity storage devices now available are hard disk drives (HDDs). However, when there is a large inventory of HDDs in the storage network, performance may suffer. That's because of the unavoidable access delays imposed by the rotational latency and time limitations inherent in the rotating platter technology used by HDDs. This has led to the perception by some that SDS is inherently slow, at least compared to implementations that employ Solid State Drives (SSDs), commonly called flash memory.
Because it is random access, has no moving parts, and therefore no latency issues, flash is the highest performance memory technology in widespread commercial use. But it is also the most costly. For most data centers, an all-flash solution would simply be cost-prohibitive.
But one of the great advantages of SDS is that it is hardware agnostic. It doesn't really care whether the memory units it manages consist of HDDs, SSDs, or a mixture of both. At the same time, the SDS software can be set up to direct access to particular storage units based on the performance requirements of specific applications. This means that the storage system can be configured with an appropriate ratio of slow but low-cost HDDs and high cost but speedy flash memory units. The SDS software can then be set up to automatically match workloads that require the highest level of performance with flash resources, while using slower HDD units for the rest.
Myth #7: SDS Is Not Ready For Prime Time
SDS is a complex technology. As Salvatore DeSimone, Vice President & Chief Technology Officer, Emerging Technologies at EMC puts it, "Designing a system that reliably stores your data with scale-out, multi-protocol access, geo distribution, in-place analytics capabilities, and with a simple management experience is a very hard problem. This is why it has taken storage vendors time to deliver on the promise."
But SDS is no longer in its infancy. The first commercial products date from about 2003. As the technology has matured, its reliability, feature set, and ease of use have grown apace. Earlier concerns about the suitability SDS to meet the mission-critical storage requirement of world-class enterprises have been put to rest.
The thousands of organizations large and small that now depend on SDS, along with many more that are planning to adopt it in the near future, are demonstrating their confidence every day that this ground-breaking technology is indeed ready for prime time.
If you'd like to explore further how Software Defined Storage can benefit your company, please take a look at the Talon FAST™ video.