Hadoop is responsible for computing a varying array of data
products at LinkedIn, including People You May Know (LinkedIn’s
people recommendation service), People Who Viewed This Also
Viewed (LinkedIn’s collaborative filtering), Who’s Viewed My
Profile?, Career Center, LinkedIn’s job recommendations, and
more. These products are immensely successful and extremely data
intensive: People You May Know, for example, generates a
significant portion of the invitations on LinkedIn, churning
through over 50 TB of data every day.
In this talk, I will detail the pieces of infrastructure that
allow us to make this happen (all open sourced), which will allow
an attendee to build their own data products. I will also give
tips & tricks that we have learned, sometimes painfully, along
the way. This talk is geared towards the intermediate Hadoop user
who perhaps has a few jobs that compute some data, but wants to
learn how to put this into a productionized process. There will
also be some nuggets for advanced users on how LinkedIn deals
with big data.
The talk will be subdivided into 4 “proverbs,” as follows.
Sam Shah is a Senior Software Engineer in the
Search, Network, and Analytics Team at LinkedIn,
working on applied data products. He is
particularly involved in the relevance backends
behind “People You May Know,” LinkedIn’s people
recommendation service, and LinkedIn’s
collaborative filtering system. He holds a Ph.D.
from the University of Michigan.
Comments on this page are now closed.
For information on exhibition and sponsorship opportunities at the conference, contact Susan Young at email@example.com
Download the Strata Sponsor/Exhibitor Prospectus
View a complete list of Strata Contacts