Object Storage vs. Block Storage

When building your cloud practice, there are three major resource components you need to consider: Compute, Networking, and Storage. Green Cloud has historically focused on robust, modern Compute and Networking resources. With the most recent addition to our product portfolio, Object Storage, we will also bring cutting-edge Storage options to our cloud partners.

Object Storage is a new, somewhat mysterious concept for many cloud providers; they likely know it exists, but may not be familiar with how it compares to other legacy data storage models. In this article we will compare Object Storage to Block Storage in order to illustrate why Object Storage excels in modern Cloud-Native architectures. We will look at the differences in design philosophy between these storage solutions, then how those design changes impact usability, and finally how those usability differences affect the use cases for which Object and Block storage are best suited.

First, what is Block Storage?

Block Storage is the data storage model with which most users are familiar. It is the easiest model to immediately grasp: your Microsoft Word documents, enterprise databases, and cat photos are all stored as files on a disk or array. These files have a beginning and end somewhere on one or more hard disks, and are chopped up into discrete units called Blocks. Each Block has an address that the Operating System knows, and that address is the only way to access that specific file (or piece of a file).

When a file transaction happens on local storage, blocks are a very effective and low-latency way to store and access those files. However, what happens when we need to access files remotely, for instance from an API call inside an application? We need to know a few things to make that happen:

  • The exact location of the file

  • The format to expect when receiving that file

  • The protocol of the server that serves us the file

If any of those parameters are incorrect, this could cause issues in our application. We could design our application around this storage paradigm to mitigate these issues… but what if we didn’t have to? With more and more workloads shifting to Cloud-Native designs that require remote data access, an opportunity appeared to redefine storage altogether.

How is Object Storage Different?

Let’s revisit our API file access example above. If we were to design data storage specifically around the needs of software engineers, we would want to solve these problems by providing:

  • Alternate ways to address and locate files (i.e., metadata)

  • Format-agnostic file delivery

  • Location-agnostic access

With these design tenets in mind, in 2006 Amazon’s AWS division defined a new API Standard called S3 (Simple Storage Service). S3 introduced a single storage unit, aptly named a “Bucket”. Buckets hold S3’s only data unit, the “Object”. To access Objects, the user needs only the following information:

  • The Bucket URL

  • The Access Keys for the Bucket

  • Metadata that applies to the Object, such as:

    • Name

    • File Format

    • Custom Metadata Tags

To illustrate this difference, let’s look at an example.

Example: Storing Dogs

Imagine we are tasked with running a dog kennel. We need a way to track where the dogs end up. If we designed our dog storage around Blocks, we would need to start with a large number of crates, and then create a list of the dogs that are contained in each crate. Were someone to spill coffee on the list, or try to find a dog without knowing its crate number, we would immediately run into trouble.

An Object Storage bucket-kennel would work a bit differently. Upon arrival, each dog would receive its own electronic tag with information like Name, Owner, Breed, Color, and so on. These tags are all entered into a database. Then, when the owner arrives, they would have many ways to locate and retrieve their dog.

To apply this example: rather than having to know the specific server, protocol, and location for a File, the software engineer only needs to know in which Bucket it is located, and what metadata is associated with the Object. Since the engineer can also define this metadata when they first store the Object, they can design the data access portion of their application to be extremely simple and easy-to-maintain.

Compare & Contrast: Object vs. Block Storage

Now that we have seen the design differences between these two storage architectures, the next question for cloud solution designers then becomes: “How do these differences affect usability?” As infrastructure providers at Green Cloud, we want to examine solutions based on their Reliability, Performance, and Scalability.

Reliability

Object Storage uses complex data protection algorithms such as Erasure Coding to abstract away the underlying Storage infrastructure. This allows Cloud providers to focus on drive integrity, which greatly improves reliability.

Performance

Block Storage will out-perform Object Storage when an application makes many individual data transactions per second, such as with large databases. For distributed data and streaming, however, Object Storage reduces the latency to retrieve data by delivering from geo-redundant locations.

Scalability

Since Block Storage is addressed, there is an automatic limitation built-in to how much data can be addressed. On the other hand, Object Storage buckets can scale quickly and infinitely through metadata tagging. For large amounts of unstructured data, Object Storage handily outperforms Block.

Comparing Use Cases

After seeing the usability differences above, it should seem clear that Object Storage shows some material improvements over Block Storage. In this section we break down several real-world use cases and draw connections to why Object Storage’s strengths make it a better fit in certain areas.

Bulk Storage Offload

Offloading NAS and Backup data to Object Storage is an obvious use case. When backup data reaches a certain age, applications like Veeam can automatically transfer data from locally-accessible Block Storage to remote Object Storage. This reduces cost and frees up space for more recent backups. Retention can also be automatically enforced on the Object Storage side through metadata tagging.

Media Assets

Media Asset Management applications organize and manage large amounts of raw media files. Because raw video takes up so much disk space, Block Storage will quickly fill up and become difficult to maintain. This becomes yet more impactful when dealing with Video Surveillance and Security, since the surveillance data must be closely tracked and maintained for long periods of time. Through high scalability and metadata tagging, Object Storage greatly improves usability for media-based applications.

Big Data

Machine Learning and Data Analysis techniques are growing and changing at an explosive rate. Algorithms built for these purposes require access to an extraordinary amount of data, both for training and processing purposes. Object Storage is the only reasonable way to provide petabytes or exabytes of contiguous storage to these Big Data applications.

Infrastructure-as-Code

Finally, modern cloud-native applications utilize an “Infrastructure-as-Code” approach. Object Storage is built to facilitate IaC by defining storage endpoints by URL and access key, which are very simple to implement programmatically. The S3 API provides a standard format by which orchestration platforms such as Kubernetes can access and manage Object Storage buckets.

Takeaways for Cloud MSPs

Object Storage represents a movement away from traditional local architecture towards distributed, infrastructure-agnostic computing. In a sense, Block Storage is never going away. Large transactional databases will likely always perform best with quick, local data storage. However, the improvements in Performance, Reliability, and Scalability that Object Storage offers cannot be overlooked, and some use-cases are a very obvious immediate fit. Green Cloud is excited to offer Object Storage to our partners as an extension of their cloud practice. If you would like to know more about Green Cloud’s new storage offerings, feel free to contact your Account Manager, or visit https://gogreencloud.com/contact-us/.

Previous