Facebook’s Large Scale Monitoring System Built on HBase
ODS is Facebook’s internal monitoring system, which collects a variety of system and application level metrics from every server for real-time monitoring, anomaly detection, alerting and analysis.
In this talk, we’ll start by giving an overview of ODS, and go over the manageability and scalability challenges with the previous MySQL based setup. We will then discuss our motivation to choose HBase for this workload, and share a series of lessons learnt from building ODS on top of HBase, including takeaways from our attempts to scale the HBase cluster separately from the HDFS cluster, a dual-HBase architecture for high availability, migration data from MySQL to HBase, running MapReduce jobs for time-based rollup of metrics.
People planning to attend this session also want to see:
Liyin Tang
Liyin Tang is a software engineer at Facebook and a HBase Committer at the Apache Software Foundation. At Facebook, he works on building the data storage system based on HBase for various applications. Liyin holds a bachelor degree in Software Engineering from Shanghai Jiao Tong University, China and a master degree in Computer Science from University of Southern California, US.
Vinod Venkataraman
Vinod Venkataraman is a Software Engineer at Facebook, where he focuses on developing the in-house monitoring systems. Vinod holds a Master’s degree from the University of Texas at Austin, and a Bachelor’s degree from the National Institute of Technology, Trichy, India, both in Computer Science.
Charles Thayer
Charles Thayer is a Software Engineer at Facebook, where he works on the Monitoring Systems. Before Facebook, he worked at Yahoo on search technology including the Web Crawler and Hosted Vertical Crawler. His focus has been scaling both storage and compute resources across thousands of nodes and tens of thousands of disks. He’s been involved with many startups in NYC including Metrobeat/Citysearch and CityRealty. He graduated with a BS EE from Columbia University’s School of Engineering and Applied Science, before becoming CEO of his first startup, Mediabridge Infosystems.
Comments on this page are now closed.
Schedule views
Sponsors
- Actian
- Amazon Web Services
- AMD
- Arista Networks
- Birst
- Booz Allen Hamilton
- Calpont
- Central Intelligence Agency
- Cignex Datamatics
- Cirro
- Cloudwick Technologies
- Continuuity
- Convey Computer
- Coraid
- Couchbase
- Datameer
- Dell
- Feedzai
- GridGain
- Hadapt
- HP
- HStreaming
- Impetus
- Karmasphere
- LucidWorks
- Open BI
- Oracle
- OrangeFS
- ParAccel
- Penguin Computing
- Pervasive Software
- Qubole
- Quest Software
- Rainstor
- Revolution Analytics
- Scaleout Software
- Simba Technologies Inc.
- SiSense
- SoftLayer Technologies, Inc.
- StackIQ
- Supermicro
- Syncsort
- Tableau Software
- Talend
- Teradata Aster
- Think Big Analytics, Inc.
- Yahoo!
For information on exhibition and sponsorship opportunities, contact Susan Stewart at sstewart@oreilly.com.
For information on trade opportunities contact Kathy Yu at mediapartners
@oreilly.com
For media-related inquiries, contact Maureen Jennings at maureen@oreilly.com
View a complete list of Strata contacts.
CO-PRESENTED BY:
(800) 889-8969 or (707) 827-7019
Monday-Friday 7:30am-5pm PT
All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners.
conf-webmaster@oreilly.com
Comments
FYI: I have uploaded the slides here.
could you please share the ppt?