ODS is Facebook’s internal monitoring system, which collects a variety of system and application level metrics from every server for real-time monitoring, anomaly detection, alerting and analysis.
In this talk, we’ll start by giving an overview of ODS, and go over the manageability and scalability challenges with the previous MySQL based setup. We will then discuss our motivation to choose HBase for this workload, and share a series of lessons learnt from building ODS on top of HBase, including takeaways from our attempts to scale the HBase cluster separately from the HDFS cluster, a dual-HBase architecture for high availability, migration data from MySQL to HBase, running MapReduce jobs for time-based rollup of metrics.
Liyin Tang is a software engineer at Facebook and a HBase Committer at the Apache Software Foundation. At Facebook, he works on building the data storage system based on HBase for various applications. Liyin holds a bachelor degree in Software Engineering from Shanghai Jiao Tong University, China and a master degree in Computer Science from University of Southern California, US.
Vinod Venkataraman is a Software Engineer at Facebook, where he focuses on developing the in-house monitoring systems. Vinod holds a Master’s degree from the University of Texas at Austin, and a Bachelor’s degree from the National Institute of Technology, Trichy, India, both in Computer Science.
Charles Thayer is a Software Engineer at Facebook, where he works on the Monitoring Systems. Before Facebook, he worked at Yahoo on search technology including the Web Crawler and Hosted Vertical Crawler. His focus has been scaling both storage and compute resources across thousands of nodes and tens of thousands of disks. He’s been involved with many startups in NYC including Metrobeat/Citysearch and CityRealty. He graduated with a BS EE from Columbia University’s School of Engineering and Applied Science, before becoming CEO of his first startup, Mediabridge Infosystems.
Comments on this page are now closed.
For information on exhibition and sponsorship opportunities, contact Susan Stewart at firstname.lastname@example.org.
For information on trade opportunities contact Kathy Yu at mediapartners
For media-related inquiries, contact Maureen Jennings at email@example.com
View a complete list of Strata contacts.