Optimising scarce resources using real-time decision making

Alasdair Allan (The Thing System, Inc.)
Real-time
Location: Sutton South

In the last few years the ubiquitous availability of high bandwidth networks has changed the way both robotic and non-robotic telescopes operate, with single isolated telescopes being integrated into expanding smart telescope networks that can span continents and respond to transient events in seconds. At the same time the rise of data warehousing has made data mining more practical, and correlations between new and existing data can be drawn in real time. These changes have led to fundamental shifts in the way astronomers pursue their science. Astronomy, once a data-poor science, has become data-rich.

For many applications it is practical to extend data warehousing to real-time assets such as telescopes. There are few real intrinsic differences between a database and a telescope other than the access time for your data and the time stamps on the data itself. Inside astronomy architectures are emerging which present both static and real-time data resources using the same interface, inherited from a superset of the functionality possessed by both types of resource.

In these architectures all the components of the system, including the software controlling the science programmes, are thought of as agents. A negotiation takes place between these agents in which each of the resources bids to carry out the work, with the science agent scheduling the work with the agent embedded at the resource that promises to return the best result.

Effectively these architectures can be viewed as a general way to co-ordinate distributed (sensor) platforms, preserving inherent platform autonomy, using collective decision making to allocate resources. Such architectures are applicable to many (geographical) distributed sensors problems, or more generally to problems where you must optimise output from a distributed system in the face of scarce resources.

This talk explores the emergence of these architectures in the astronomical community from the viewpoint of one of the people intimately involved in the process. The talk will walk attendees through the pitfalls faced by developers hoping to implement such novel architectures and discuss how the deployment of these architectures in the field has prompted the interesting and increasing use of scientists as mechanical turks by their own software.

Photo of Alasdair Allan

Alasdair Allan

The Thing System, Inc.

Alasdair Allan is the author of Learning iPhone Programming and iPhone Sensor Programming published by O’Reilly Media. He is a senior research fellow in Astronomy at the University of Exeter, and as part of his work there he is building a distributed peer-to-peer network of telescopes that, acting autonomously, can reactively schedule observations of time-critical events and carry out complex long term monitoring of variable objects. Notable successes include contributing to the detection of the most distant object yet discovered, a gamma-ray burster at a redshift of 8.2. Alasdair also runs a small technology consulting company writing bespoke software, building open hardware and providing training. He sporadically writes blog posts about things that interest him, and more frequently provides commentary about them in 140 characters or less.

Sponsors

  • Aster Data
  • EMC Greenplum
  • GE
  • Lexis Nexis
  • MarkLogic
  • Tableau Software
  • Cloudera
  • DataStax
  • Informatica
  • DataSift
  • Splunk
  • Amazon Web Services
  • Datameer
  • Impetus
  • Karmasphere
  • MapR Technologies
  • Pervasive
  • Platform Computing
  • Revolution Analytics
  • Sybase
  • Xeround
  • Media-Science
  • Platfora

Sponsorship Opportunities

For information on sponsorship opportunities at the conference, contact Susan Stewart at sstewart@oreilly.com

Press & Media

For media-related inquiries, contact Maureen Jennings at maureen@oreilly.com

Contact Us

View a complete list of Strata Contacts