Large scale web mining
This tutorial will teach attendees about the key aspects of scalable web mining, via six modules:
1. Introduction- Why web data is valuable
- Key challenges to web crawling
- Realistic definitions for success
- Reducing time & cost by focusing the crawl
- Approaches to classifying and scoring pages
- Solutions for scalable web crawling
- Data mining essentials
- Structured text extraction
- Automated vs. manual extraction
- Making it searchable
- Finding “interesting” text
- Machine learning with Mahout
- Polite crawling versus deep crawling
- Spam, splog, honeypots and nasty webmasters
- Ajax, robots.txt and Facebook
- Hotel reviews * Music pages * SEO analysis
People planning to attend this session also want to see:
Ken Krugler
Scale Unlimited
Veteran developer and entrepreneur, 25+ years experience. Founder and President of TransPac Software, a 20 year leader in internationalization, mobile devices, and search consulting. Founder and CTO of Krugle, a vertical search engine and enterprise appliance for code and technical information. Co-founder of Bixo web mining project. Committer for the Apache Tika project. Author and speaker on vertical search and web mining.
Comments on this page are now closed.
Schedule views
Sponsors
For information on exhibition and sponsorship opportunities at the conference, contact Susan Stewart at sstewart@oreilly.com.
For information on trade opportunities with O'Reilly conferences contact Kathy Yu at mediapartners
@oreilly.com
For media-related inquiries, contact Maureen Jennings at maureen@oreilly.com
View a complete list of Strata contacts
©2011, O'Reilly Media, Inc.
(800) 889-8969 or (707) 827-7019
Monday-Friday 7:30am-5pm PT
All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners.
conf-webmaster@oreilly.com
Comments
Hi Jim – Mac is actually the preferred platform. Windows users need to install Cygwin to get a Linux-like environment for running Hadoop, but Mac already has that – you can just use the regular Terminal window.
The instructions for lab appear to assume a windows user. Are the tools required already installed for a mac? Is there a preference on a windows vs mac set up?