Skip to main content

Deep Learning and the Dream of AI

Brandon Ballinger (Independent )
Hardcore Data Science Gramercy Suite
Average rating: ***..
(3.57, 7 ratings)
Slides:   1-PDF 

There’s been a lot of fuss over deep learning in the last year, and for good reason: deep neural networks (also called deep belief networks) have beat conventional algorithms in applications as diverse as malware detection, speech recognition, computer vision, and molecular activity prediction. The idea of neural networks is hardly new - pioneering work in the 50’s and 70’s introduced the widely-used perceptron and back-propagation algorithms - but today’s neural networks can efficiently process many more neurons, with many more layers, than before.

What makes today’s neural networks so much more powerful? It’s a combination of hardware, algorithms, and implementation. First, due to Moore’s law, today’s hardware can accommodate much larger neural networks than in the past. Second, in 2006, Geoff Hinton introduced a new algorithm - greedy layer-wise pre-training - which allows for efficiently training larger and deeper neural networks in the past. Finally, the best implementations today make use of GPUs to speed up training.

Deep learning has succeeded in two recent state-of-the-art applications. Recently, researchers from University of Toronto won a Kaggle competition for molecular activity prediction using deep neural networks. Furthermore, Google has switched Android’s speech recognition engine to use deep neural networks.

Finally, we’ll wrap up by giving a quick summary of what was covered and some recent work in the field.

Outline:
  • Overview: What are deep neural networks? What will we cover?
  • Applications: new records set by deep belief networks.
  • Background: Classical neural networks from the 60’s and 80’s
  • What’s different now: * Faster hardware * Greedy layer-wise pre-training. * GPU implementation (Matlab / Python)
  • Techniques: rectified linear units, dropout, etc.
  • Example application: molecular activity prediction (Kaggle)
  • Example application: speech recognition (Google/Android)
  • Quick overview of recent work in the field.

Brandon Ballinger

Independent

Brandon Ballinger has applied machine learning practitioner to fraud detection, speech recognition, online advertising, and more. He co-founded of Sift Science, a San Francisco startup that uses large-scale machine learning to fight online fraud. Before that, he was a Staff Software Engineer at Google working on Android speech recognition. He studied Computer Science at the University of Washington where he worked on research projects in computational mathematics, network congestion control protocols, and numerical models of air-sea coupling.

Sponsors

Sponsorship Opportunities

For exhibition and sponsorship opportunities, contact Susan Stewart at sstewart@oreilly.com

Media Partner Opportunities

For information on trade opportunities with O'Reilly conferences email mediapartners
@oreilly.com

Press & Media

For media-related inquiries, contact Maureen Jennings at maureen@oreilly.com

Contact Us

View a complete list of Strata + Hadoop World 2013 contacts