From Predictive Modelling to Optimization: The Next Frontier

Jeremy Howard (Kaggle)
Data Science, Mission City B1
Average rating: ****.
(4.50, 4 ratings)

In “The Evolution of Data Products”, O’Reilly Media’s Mike Loukides notes: “the question of how we take the next step — where data recedes into the background — is surprisingly tough. Do we want products that deliver data? Or do we want products that deliver results based on data? We’re evolving toward the latter, though we’re not there yet.” In this talk, Jeremy Howard will show why taking this step is tough, and will lay out what needs to be done to deliver results based on data. He will particularly draw on his experience in building Optimal Decisions Group, where he developed a new approach to insurance pricing which focused on delivering results (i.e.: determine the optimal price for a customer) instead of delivering data (i.e. calculating a customer’s risk, which had been the standard approach used by actuaries previously).

Delivering results based on data requires 3 steps:

1) Creating predictive models for each component of the system 2) Combining these predictive models into a simulation 3) Using an optimization algorithm to optimize the inputs to the simulation based on the desired outcomes and the system constraints

Unfortunately, many data scientists today are not sufficiently familiar with steps 2) and 3) of this process. Although many data scientists have been developing skills in predictive modelling, simulation and optimization skills are still rare. Jeremy will show how these 3 steps fit together, give examples of their use in real world situations, and will introduce some of the key algorithms and methods that can be used.

Photo of Jeremy Howard

Jeremy Howard

Kaggle

Jeremy Howard is President and Chief Scientist at Kaggle. Previously, he founded FastMail (sold to Opera Software) and Optimal Decisions sold to ChoicePoint – now called LexisNexis Risk Solutions). Prior to that he worked in management consulting, at McKinsey & Company and A.T. Kearney. Jeremy’s passion is applying algorithms to data. At FastMail he used algorithms to automate nearly every part of the business – as a result the company only needed a total of 3 full time staff, and got over a million signups. Optimal Decisions was a business entirely built to commercialise a new algorithm he designed for the optimal pricing of insurance. Jeremy competes regularly in data mining competitions, which he uses to test himself and stay on the leading edge of machine learning and predictive modelling technology. He is currently ranked #1 on Kaggle’s overall competitor rankings, out of over 16,000 data scientists.

Comments on this page are now closed.

Comments

Picture of Dirk Van den Poel
Dirk Van den Poel
03/01/2012 12:30pm PST

See our 2004 paper using a GA-based marketing optimization system. It has been implemented as an HPC-cluster application to speed up computations. See: www.crm.ugent.be/marketingo... or send me an e-mail: dirk.vandenpoel@ugent.be

Sponsors

  • EMC
  • Microsoft
  • HPCC Systems™ from LexisNexis® Risk Solutions
  • MarkLogic
  • Shared Learning Collaborative
  • Cloudera
  • Digital Reasoning Systems
  • Pentaho
  • Rackspace Hosting
  • Teradata Aster
  • VMware
  • IBM
  • NetApp
  • Oracle
  • 1010data
  • 10gen
  • Acxiom
  • Amazon Web Services
  • Calpont
  • Cisco
  • Couchbase
  • Cray
  • Datameer
  • DataSift
  • DataStax
  • Esri
  • Facebook
  • Feedzai
  • Hadapt
  • Hortonworks
  • Impetus
  • Jaspersoft
  • Karmasphere
  • Lucid Imagination
  • MapR Technologies
  • Pervasive
  • Platform Computing
  • Revolution Analytics
  • Scaleout Software
  • Skytree, Inc.
  • Splunk
  • Tableau Software
  • Talend

For information on exhibition and sponsorship opportunities at the conference, contact Susan Stewart at sstewart@oreilly.com.

For information on trade opportunities with O'Reilly conferences contact Kathy Yu at mediapartners
@oreilly.com

For media-related inquiries, contact Maureen Jennings at maureen@oreilly.com

View a complete list of Strata contacts