Skip to main content

The IPython Notebook: Get Close to Your Data with Python and JavaScript

Brian Granger (Cal Poly San Luis Obispo)
Design
GA Ballroom K
Average rating: ****.
(4.56, 16 ratings)

The web, through powerful libraries such as d3.js, is transforming how we work with data. However, the browser is a long way from where most of our code/data actually lives, namely, on the server side. To leverage the power of JavaScript/HTML/CSS, users must develop custom web applications for each problem. This is much too painful for exploratory data science, where the end goal is a moving target.

In this talk I will describe how the IPython Notebook solves this problem by reducing the distance between a user and their code/data. This is accomplished by a new widget architecture that makes it easy to leverage the power of both JavaScript/HTML and Python in exploratory data science.

The IPython Notebook is an open-source, web-based interactive computing environment for Python, R, shell scripts, Julia and other languages. At its core, the Notebook is an environment for writing and running code in an interactive and exploratory manner. On top of this foundation, it adds a document based workflow: Notebook documents contain live code, descriptive text, mathematical equations, images, videos and arbitrary HTML. These documents provide a complete and reproducible record of a computation and can be shared with others, version controlled and converted to a wide range of of static formats (HTML, PDF, slides, etc.).

After reviewing the basics of the IPython Notebook, I will describe its underlying architecture and the recent extensions to support interactive JavaScript/HTML widgets. The IPython architecture consists of separate server-side processes, called Kernels, that run user’s code (in Python, R, Julia, etc.) and returns output to the browser over a JSON based message protocol (over WebSockets and ZeroMQ). This message protocol now includes the ability to synchronize the state of JavaScript objects living in the browser to computations and data living in the Kernel. The result is that data scientists doing exploratory work can easily leverage both modern web technologies, such as d3.js, and the computational capabilities of languages such as Python, R and Julia in a single integrated environment. Examples of applications of these capabilities include:

  • Parameters in Kernel side computations can be coupled to UI controls (sliders, drop down menus) in the browser.
  • Browser side visualizations can be backed by Kernel side data structures (Data Frames, networks/graphs, time series) and their states can be synchronized based on user interactions.
  • Kernel side data structures can be edited and created using browser side UIs.

I will describe the widget architecture in detail and then show examples of how it can be used in exploratory work. The talk will be of interest to developers interested in using the architecture to build widgets and for working data scientists who could use those widgets. I will also briefly mention how other backend languages, such as Julia and R can leverage this same architecture.

Photo of Brian Granger

Brian Granger

Assistant Professor, Cal Poly San Luis Obispo

Brian Granger is an Assistant Professor of Physics at Cal Poly State
University in San Luis Obispo, CA. He has a background in theoretical
atomic, molecular and optical physics, with a Ph.D from the University of Colorado. His current research interests include quantum computing, parallel and distributed computing and interactive computing environments for scientific and technical computing. He is a core developer of the IPython project and is an active contributor to a number of other open source projects focused on scientific computing in Python. He is @ellisonbg on Twitter and GitHub.