Open Source Ceph Storage– Scaling from Gigabytes to Exabytes with Intelligent Nodes

As the size and performance requirements of storage systems have increased, file system designers have looked to new architectures to facilitate system scalability.

Ceph’s architecture consists of an object storage, block storage and a POSIX-compliant file system. It’s in the most significant storage system that has been accepted into the Linux kernel. Ceph has both kernel and userland implementations.The CRUSH algorithm controlled, scalable, decentralized placement of replicated data. In addition, it has a fully leveraged, highly scalable metadata layer. Ceph offers compatibility with S3, Swift and Google Storage and is a drop in replacement for HDFS (and other File Systems).

Ceph is unique because it’s massively scalable to the exabyte level. The storage system is self-managing and self-healing which means limited system administrator involvement. It runs on commodity hardware, has no single point of failure, leverages an intelligent storage node system and it open source.

This talk will describe the Ceph architecture and then focus on the current status and future of the project. This will include a discussion of Ceph’s integration with Openstack, the file system, RBD clients in the Linux kernel, RBD support for virtual block devices in Qemu/KVM and libvirt, and current engineering challenges.