The Yin & Yang of Software-Defined Storage
by Jaap van Duijvenbode on January 18, 2017
As the amount of data corporations must deal with continues to explode, traditional storage solutions have had a hard time keeping up. Normal data center practice has been to simply purchase additional storage hardware whenever currently installed systems begin to hit their capacity ceilings. But this approach has become increasingly costly and difficult to manage.
That's been one of the driving forces behind the increasing penetration of software defined storage (SDS) into corporate data centers. As the promise of SDS has become more apparent, almost every storage manufacturer has begun offering SDS products, and corporate IT managers are faced with the question of whether SDS is a viable option for them.
What Is SDS?
SDS is still a relatively new paradigm, and there is as yet no settled, precise definition of the term. There is, however, broad agreement about the overall concept.
An SDS storage system is one in which the data store is virtualized and managed through software. That software layer presents a consistent interface to users (and to using applications, via APIs), without regard to the nature, configuration, and even geographical distribution of the physical storage resources. With this approach, a storage infrastructure that may include many different types of devices and media, located at different points around the world, can be accessed and controlled as a single, unified pool of storage.
How SDS Compares With Traditional Storage Solutions
In the traditional data center, storage has been implemented using proprietary hardware arrays built using custom-designed ASICs (Application-Specific Integrated Circuits) and controlled by a custom-designed real-time operating system. All that custom designing adds up to a lot of expense. The ASICs alone can cost upwards of $10 million to develop and produce. And because the entire system is proprietary, storage administrators have little flexibility and can easily find themselves locked-in to a particular vendor.
SDS systems, on the other hand, are designed to be hardware-agnostic. Because the intelligence of the storage system resides in software rather than in the individual storage devices, it doesn't really matter what those devices may be. Most SDS offerings are based on the industry standard x86 architecture, which has decades of development and use behind it. Theoretically, SDS can turn generic x86 servers into storage systems. So, rather than being dependent on costly proprietary storage units, an SDS system can employ inexpensive commodity hard disk drive (HDD) arrays and servers.
This gives SDS implementations an advantage over the traditional storage approach in several areas.
SDS Acquisition Costs Are Lower
The ability of SDS offerings to use COTS x86 servers and commodity HDDs rather than custom-designed, proprietary storage appliances reduces hardware acquisition costs significantly. Plus, the ability of SDS systems to handle a mix of hardware and media types allows data centers to leverage their existing storage units under the SDS umbrella.
SDS Increases Storage Management Efficiency
One of the greatest advantages of SDS over the traditional approach to storage is that the SDS software is able to control the entire storage pool through automated, policy-based management processes. Administrators are relieved of the necessity of handling each storage device separately. Common storage functions such as deduplication, replication, and snapshots can be managed on a system-wide basis, rather than by having to individually configure each device or subsystem.
The ability SDS provides to monitor and oversee the system as a whole also contributes to a dramatic increase in resource utilization. With traditional storage architectures, utilization of storage resources normally tops out at about 30 percent. But SDS implementations can effectively double that rate.
SDS Provides Greater Flexibility
Because SDS handles all storage units as virtual devices, provisioning need no longer be a matter of physically attaching and configuring new hardware. That task can be handled, often automatically, by the SDS software, which can simply add virtual resources as needed.
Scalability is another area in which SDS provides greater flexibility than traditional implementations. According to Julien Quintard, CEO at Infinit, "Software-defined solutions can scale up to several petabytes (even to exabytes if well-architectured) whereas appliance-based solutions are inherently limited in terms of storage capacity. In addition, software-based solutions can very quickly scale out to thousands of computer nodes."
SDS Enhances Data Security
A very important advantage of SDS relates to data security. In an SDS environment, standard information protection tools, such as data backup, replication, encryption, and disaster recovery procedures, can be applied and managed through a unified software interface, rather than having to be applied to each storage unit or subsystem. Data protection best practices can be easily implemented throughout the organization's storage infrastructure from the SDS console, or through automated, policy-driven processes. That allows for a much more coherent and effective data security regime than is the norm with traditional storage solutions.
SDS May Be Harder To Implement
From an acquisition cost perspective, the way to maximize the benefits of SDS over a traditional approach is to purchase a software-only SDS product, and marry it with generic x86 hardware. This allows, for example, use of inexpensive commodity HDDs, as well as re-use of legacy storage arrays that are already installed in the data center.
The problem is that even though all of the hardware components may be built to the common x86 standard, each device will still have its own unique characteristics that must be taken into account in assembling a high performance storage system. That's a system integration task, and not a trivial one. George Crump, president of Storage Switzerland, notes that the very flexibility offered by SDS can make it more difficult and time-consuming to implement. "While [SDS] does bring flexibility," he says, "it provides a storage architecture designer with more options. This translates into a much more extensive hardware selection process." As Laz Vekiarides, chief technology officer at ClearSky Data puts it, "Getting high performance and availability requires a lot of tuning."
On the other hand, in the traditional storage model, customers can buy a storage appliance that bundles hardware and software, and is essentially ready to go out of the box.
In addition, when a storage subsystem is purchased as a unit from a single provider, support issues are minimized. If a customer purchases SDS software from one vendor, and combines it with hardware from perhaps several other vendors, obtaining adequate support can be problematic. Although each provider may do a great job of supporting its own products, who takes ownership to resolve issues involving the integrated system as a whole?
SDS Is Leading The Way To The Software Defined Data Center
The shift from the traditional storage paradigm toward the software-defined concept is gathering momentum. For many storage administrators, adopting SDS will be a first step into the data center of the future, in which all IT infrastructure functions, including networking, security, and storage, are virtualized and delivered as software-controlled services. Vangie Beal, writing for webopedia.com, describes that coming software defined data center (SDDC) this way:
"Control of the data center is fully automated by software, meaning hardware configuration is maintained through intelligent software systems. This is in contrast to traditional data centers where the infrastructure is typically defined by hardware and devices."
The transition from traditional storage to SDS and eventually the SDDC is already well under way. If you'd like to know more about how to begin that transition in your own data center, we'd be happy to help. Please visit our Next Generation Software-defined Storage page.