HDFS - What is New and Future

Sanjay Radia (Hortonworks), Todd Lipcon (Cloudera, Inc.)
Hadoop: Tools & Technology, Grand East (NY Hilton)
Average rating: ****.
(4.57, 7 ratings)

Hadoop 1.0 is a significant milestone in being the most stable and robust Hadoop release tested in production against a variety of applications. It offers improved performance, support for HBase, disk-fail-in-place, Webhdfs, etc over previous releases. The next major release, Hadoop 2.0 offers several significant HDFS improvements including new append-pipeline, federation, wire compatibility, NameNode HA, further performance improvements, etc. We describe how to take advantages of the new features and their benefits. We also discuss some of the misconceptions and myths about HDFS.

The second half of the talk describes our plans for HDFS over the next year. This includes improvements such as Snapshots, Disaster recovery, RAID, performance, scaling, etc.

Sanjay Radia

Hortonworks

Sanjay is an Apache Hadoop committer and member of the Apache Hadoop PMC.
Prior to co-founding Hortonworks, Sanjay was the chief architect of core-Hadoop at Yahoo focusing mostly on HDFS, MapReduce schedulers, compatibility, etc.
He has also held senior engineering positions at Sun Microsystems and INRIA, where he developed software for distributed systems and grid/utility computing infrastructures.
Sanjay has a PhD in Computer Science from the University of Waterloo in Canada.

Todd Lipcon

Cloudera, Inc.

Todd holds a Sc.B in Computer Science from Brown University, where he completed an honors thesis developing a new collaborative filtering algorithm for the Netflix Prize Competition. Todd interned at Google developing machine learning methods to detect credit card fraud on AdWords and Google Checkout. Currently Todd works at Cloudera, Inc on bringing Map/Reduce technology to enterprises.

Comments on this page are now closed.

Comments

Todd Lipcon
10/29/2012 6:18pm EDT

Thanks, Charles. I know we will be posting the slides on the Cloudera site soon. I’ll update here with a link when it’s available.

Charles Choi
10/29/2012 6:03pm EDT

I enjoyed your presentation. Can you please post your slide deck?

Sponsors

Sponsorship Opportunities

For information on exhibition and sponsorship opportunities, contact Susan Stewart at sstewart@oreilly.com.

Media Partner Opportunities

For information on trade opportunities contact Kathy Yu at mediapartners
@oreilly.com

Press and Media

For media-related inquiries, contact Maureen Jennings at maureen@oreilly.com

Contact Us

View a complete list of Strata contacts.