Trends in Data Storage
by Jaap van Duijvenbode on November 2, 2017
We're in the midst of a data storage revolution. Whereas data storage technologies remained relatively unchanged for decades, the last several years have witnessed the introduction to the mass market of a new series of data storage solutions that provide more flexible, cost-efficient and scalable storage options.
The data storage revolution can be easy to miss. To many people, data storage is not exactly the most exciting part of computing.
Yet an understanding of the technological advances that are reshaping the data storage ecosystem is essential in order to store data efficiently and securely. To that end, keep reading for an overview of the innovative data storage trends that are now available for everyday business use.
Background: Block Storage
From the 1950s through the 2000s, the fundamentals of data storage remained more or less the same. They were oriented around block storage, in which data is written and read in a straightforward way on physical devices.
Of course, the types of devices varied from tape archives to floppy disks, to hard disks. So did storage capacity (the first hard disk, created in 1956, supported a mere 3.75 megabytes of storage space) and read/write speeds.
Yet the core processes and features of data storage remained the same. Practices such as storage tiering were primitive or did not exist. The ability of storage to scale was strictly limited by the availability of physical storage infrastructure. All data was subject to the same levels of accessibility. The only significant variable in data cost was the type of data storage device you used, not the way you used it.
Modern Storage Trends
Over the past decade, a number of new storage techniques have become available for everyday use by businesses. These techniques improve upon block storage by providing ways to make it more flexible and efficient or offer alternatives to traditional block storage.
Some of these technologies have existed for a long time, but it has only been in the past decade or so that they have become inexpensive enough to receive massive adoption.
The storage trends that are reshaping the way data is stored, modified and analyzed today include the following...
Software-defined storage, or SDS, is a technique that makes it possible to abstract the underlying storage hardware away from data.
With SDS, it doesn't matter whether the physical devices that are storing your data are traditional hard disks or flash storage. Nor does it matter how large each device is, how the devices are partitioned or (in many cases) even which file system they are formatted with.
The result is a more flexible storage experience with a lower management burden placed on the user. Users can simply drop data into an SDS pool and retrieve it when they need it. They don't have to worry about partition tables, formatting and so on.
In addition, SDS can make it easy to build automated data replication or backup into your storage, without having to configure a RAID array (redundant array of independent disks) or other data backup framework.
Software-defined storage has enabled new storage practices in which data is distributed across many servers or nodes, rather than being stored on a single pool of disks.
With scale-out storage, the storage capacity can be easily increased by adding new disks, without having to shut down a storage pool.
Scale-out storage also makes storage more reliable because data can be spread in a redundant way across multiple nodes. That way, a failure in one node does not mean that data is lost or ceases to be accessible. The data storage cluster can instead recover automatically.
Solid-state disk, or SSD, storage is delivered via a special kind of hard drive that uses flash storage instead of magnetic storage. SSD has been around for quite a while, but it was not until the mid-2000s that SSD became affordable enough to be used for everyday storage needs on personal computers and servers.
The advantages of SSD are two-fold. First, because flash storage can be accessed more quickly than data stored on a magnetic disk platter, SSD delivers significant performance boosts. Exact performance varies depending on the type of SSD device you use, how it is partitioned and other factors, but in general SSD throughput is several times faster than that of traditional hard disks.
Second, SSD devices are more durable because they don't have moving parts. They are not immortal; flash memory will wear out eventually. However, compared to a traditional hard disk, which relies on a spinning platter to read and write data, SSD devices have a much better lifespan.
SSD storage provides a high-performance storage option that is also more reliable than traditional storage devices.
Cloud storage may not seem like a particularly innovative trend. Organizations have been using the cloud to store data for many years.
Yet what is new is how diverse and sophisticated cloud storage has become. Public cloud providers like AWS offer multiple types of cloud storage, such as S3 and EBS storage volumes.
Cloud storage has also evolved to help meet compliance challenges through solutions like the AWS GovCloud, which makes it easy to ensure that data is stored in a specific geographic region.
In the past, all data was stored in the same way, no matter how the data was used.
This was inefficient from a cost standpoint because the costs were the same to store a given pool of data whether you accessed that data on a frequent basis, or only a few times a year.
Today, the concept of tiered storage makes it easier to align storage purposes with costs. With tiered storage, you can choose from different types of storage options -- which are generally referred to as "hot," "cool" and "cold" storage -- depending on how often you need to access data, the read/write requirements of the data and so on.
Hot storage is ideal for data that needs to be accessed or overwritten frequently. In contrast, cold storage provides a cost-efficient way to store static data that you access on an occasional basis -- such as patient records that you have to store for a long period to meet compliance requirements.
Most public cloud providers offer predefined storage tiers. Azure Blob Storage is available in hot, cool and cold versions, for example.
But you don't have to use the cloud to take advantage of tiered storage. In a general sense, any storage solution in which storage media and performance is tailored to specific storage needs is a form of tiered storage.
Traditionally, data has been stored in relational databases. In a relational database, each piece of data has a specific home, and data is rigidly structured. MySQL is an example of a popular relational database.
Increasingly, organizations are now turning instead to NoSQL databases. Under the NoSQL model, databases are more flexible.
The difference can be illustrated like this: A NoSQL database is like an expandable sac into which you can toss any item of any size (within limits), whereas a relational database is like a cabinet with many small shelves inside. If you can't fit your item on a specific shelf, you have to reorder the entire cabinet in order to find a way to add your item to it.
NoSQL is, therefore, a better solution in situations where the types and size of items that you need to store are unpredictable, or where your data storage needs will scale quickly.
NoSQL is not a specific database implementation; the term refers instead to a type of database architecture. Databases that implement the NoSQL model include MongoDB and Redis. (Note, too, that there are subsets of different types of NoSQL databases; not all NoSQL databases use the same type of approach to data structures and storage. The ins-and-outs of NoSQL storage are beyond the scope of this article, however.)
Compared to a decade ago, the storage solutions available to organizations today are considerably more scalable and flexible. They also provide opportunities to make storage more cost-efficient and to gain significantly better performance.
To leverage the benefits that all of these new storage technologies provide, organizations require a storage-agnostic software platform. Storage-agnosticism is essential because not all new storage technologies are compatible with one another, and not all can be directly integrated to build a single storage stack for your organization.
Talon FAST provides a storage-agnostic platform that can support all of the technologies described above, enabling your business to take advantage of the latest storage innovations. To learn more, check out the Talon FAST video today.