DREAM Challenges - overview
DREAM Challenges pose fundamental questions about systems biology and translational science. Designed and run by a community of researchers from a variety of organizations, our challenges invite participants to propose solutions — fostering collaboration and building communities in the process. The DREAM Challenges community share a vision allowing individuals and groups to collaborate openly so that the “wisdom of the crowd” provides the greatest impact on science and human health.
The DREAM Challenges are a crowdsourcing project designed to evaluate model predictions and pathway inference algorithms in systems biology. As its ancestor Critical Assessment of Protein Structure Prediction (CASP), DREAM Challenges are structured in the form of challenges, that comprise open problems presented to the community, whose solutions (the “gold standards”) are known to the organizers but not to the participants. Participants submit their predictions of the solutions to the challenges, which are evaluated by the organizers and eventually discussed in a conference after which data, predictions and gold standards are published in a peer-reviewed journal and made openly available to the community.
The DREAM Challenges aim to address fundamental questions in systems biology, bridging theory and experimentation. The questions posed by DREAM Challenges are not simple; to deduce the structure of a biological network based on experimental data, for example, researchers test a variety of algorithms. DREAM Challenges allows researchers to compare the strengths and weaknesses of these methods and provides a sense of reliability. It is interesting to note that the combined community predictions are extremely robust and usually the most accurate DREAM Challenges highlight the areas where systems biology methods for a mechanistic understanding of biological systems are adequate and those that need to be revised. DREAM Challenges prove that rigorous scrutiny of scientific research based on community involvement is possible.
The DREAM Challenges were developed by IBM Researcher, Gustavo Stolovitzky in collaboration with Andre Califano from Columbia University in 2006. Under the direction of Dr. Stolovitzky, DREAM has grown and thrived over the succeeding 10 years as teams of scientist collaborate from universities, the public sector, and industry. In February 2013, and based on the success of collaborating to run the 2012 Sage Bionetworks/DREAM Breast Cancer Prognosis Challenge, Sage Bionetworks merged its open science efforts with those of the DREAM Challenges.
IBM Research plays a key role in the ongoing DREAM Challenges. We work with the data contributors to shape the challenge questions; to test the challenges by acting as test participants testing weather we can solve them; to develop the scoring metrics and score the submissions; and we contribute analytics to participant’s publications post-challenge. IBM Researchers actively participate in the DREAM Challenge organizing team lending their talents to shaping DREAM Challenges into a leader in crowdsourcing in life sciences.