Operating a small-size Hadoop cluster is a calm walk in a forest, while working with a big-size Hadoop cluster is a big adventure in a real jungle. The bigger elephant is, the more love and care it demands and we have discovered it in a hard way.
In this talk, we will take you for a trip into Hadoop jungle at Spotify to show the most interesting, exciting and surprising places where we have been to while growing fast from a 60 to 690-node Hadoop cluster. We will expose our JIRA tickets, real graphs, statistics, even excerpts from our dialogues. We will also share the mistakes that we made and describe the fixes that finally domesticated this love-demanding yellow elephant and its friends.
Adam Kawa works as Data Engineer at Spotify and Hadoop instructor at Compendium (Authorized Cloudera Training Partner).
He is a frequent speaker at HUGs, and the coorganizer of Warsaw and Stockholm HUGs. He blogs about Hadoop at HakunaMapData.com.
Comments on this page are now closed.
For exhibition and sponsorship opportunities, contact Susan Stewart at firstname.lastname@example.org
For information on trade opportunities with O'Reilly conferences email mediapartners
For media-related inquiries, contact Maureen Jennings at email@example.com
View a complete list of Strata + Hadoop World 2013 contacts