Ceph: Baby Steps

As a cloud enthusiast, I have had a keen interest in cloud computing, with focus on cloud infrastructure for quite a while since last year. Since the start of the year, I looked into open-source storage platforms such as Ceph, Longhorn and Minio. Longhorn: block storage; Minio: object storage; Ceph: object storage. What’s fascinating about Ceph is that it’s not limited to only object storage; it provides both block and file storage! Let’s know a little more about Ceph.

What is Ceph?

Ceph is an open-source storage platform designed to store data in multiple formats. This serve as an alternative to other cloud object storage such as AWS’ S3 and Azure’s Blob Storage. Think of it as a Swiss Army knife. You want to use Ceph as a block storage? Use Ceph RBD. Ceph as a file storage? Use CephFS. These are achieved through storage clients, with Ceph RGW being the default storage client for Ceph, serving as object storage for your system.

For S3 and other cloud storage services by the various providers, object storage is enabled by a flat architecture, where data files are stored as objects and kept in a single -and very large- repository. Ceph has a different approach entirely. Data are stored as objects in object storage devices (OSDs). The architecture utilizes RADOS (Reliable, Autonomous, Distributed Object Storage), a software-defined distributed storage that offers file, block and object storage. This software allows for capabilities such as scalability, HA reliability, and flexibility and customization. This software facilitates Ceph architecture through nodes and clusters.

Ceph Architecture

Ceph architecture is comprised of nodes and clusters. The clusters are a collection of 5 ‘maps’ each, that function as blueprints for meeting the user needs. These maps keep record of the current state of Ceph storage by updating both Ceph clients and Ceph OSDs with current information on a cluster’s topology. These maps are: the monitor, OSD, PG (Placement Group), CRUSH, and MDS maps. Every map maintains a history of changes in the operating state.

The cluster consists of 4 daemons, working interdependent of each other:

1. Ceph monitor- maintains a copy of the cluster map, provided to the storage clients.
2. Ceph OSD (Object Storage Device) Daemon- state checker for all OSDs in the cluster, including itself, and reports back to the Ceph monitor. All OSDs control read, write, and replication operations on storage drives.
3. Ceph Manager- point of monitoring, management and orchestration of the entire cluster.
4. Ceph Metadata Server- manages file metadata when using CephFS.

The nodes are physical servers running Ceph workloads. They may be storage nodes, manager nodes, monitor nodes, and metadata nodes.

With its non-centralized architecture and its distributed nature, Ceph uses the CRUSH algorithm to dynamically store data across the cluster, facilitating horizontal scaling with no single point of failure. This ensures even data distribution across all nodes in the clusters, balancing the workloads, and fault tolerance through replication, hence data redundancy.

Ceph Clients

These clients act as the storage options available to the end-user. They include:

• Ceph FS- provides a POSIX-compliant filesystem that can be mounted on various servers.
• Ceph RBD (Raw Block Device)- provides thin-provisioned, resizable block volumes.
• Ceph RGW- basically object stoage; provides RESTful APIs, with compartible interfaces to AWS’ S3 and OpenStack Swift.

Conclusion

Ceph is interesting to learn. I find that one might have to be familiar with the basics of how object, block and file storage work. You can also set up a disk partition with Ceph as its file system (played around with it on an Ubuntu server a while back. I’ll have to create the disk partition in the course of the year)! You can check the official documentation on Ceph that go into detail on what Ceph is about.

Thank you for taking your time to read this short article. I hope you have learnt something new, and possibly, ignited your interest in cloud infrastructure.