MIMIA'19 - overview

Medical Informatics in Medical Image Analytics (MIMIA’19)
A MICCAI 2019 Tutorial

October 13, 2019, Shenzhen, China

With recent advances in AI in medical imaging fueling the need to curate large, labeled datasets, first movers such as NIH, MIT, and Stanford are leveraging natural language processing (NLP) techniques to mine free labels from imaging reports, and compiling publicly available datasets to help the development and evaluation of image analytics models that will ultimately assist clinicians to make better diagnostic decisions for patients. To date, major efforts on NLP-powered data harvesting have been focused on a handful of diseases and imaging modalities, most notably the 14 common findings on chest X ray images. Extending the capabilities of known clinical NLP tools, and introducing these tools to the MICCAI community with an emphasis on a better support for local customization, are expected to speed up the label curation progress on a broader spectrum of disease findings and a wider coverage of imaging modalities and subspecialties. 

Since the initial release of the imaging labels automatically derived from associated reports, researchers have raised concerns on the accuracy of these labels and how useful they are for image analysis. While the clinical NLP community is continuously elevating the performance of label curation to the next level, additional efforts could be made towards learning from less perfect image labels. There has been long-standing interest in weakly-supervised learning with partial/uncertain labels in AI research. Successful attempts are reported in recent publications on medical imaging data, and we expect more of this line of explorations in the future. An in-depth review of traditional and recently-developed weakly-supervised learning approaches, in particular their successful applications on medical imaging data, would help imaging researchers develop algorithms that could potentially thrive on silver-standard labels.

In addition to imaging data and their corresponding reports, other varieties of multimodal retrospective patient data are also available, such as structured EMR, time series, genomics data, and so on. Analyzing these multimodal sources for disease-specific information across patients can reveal important patterns that can potentially benefit computer-aided clinical decision making. The latter is still very much an art due to the lack of quantitative tools up to accommodate multimodal patient data. Despite the tremendous development of medical image analytics algorithms in the MICCAI community, there is little collaboration between medical imaging researchers and those in the medical informatics domain. The translation of multimodal data fusion and joint learning techniques to clinical practice has been slow. Bringing together researchers from both fields and having them share ideas and insights along with clinicians, many creative sparks could fly.

The main themes of the proposed tutorial will include:

  • NLP-powered data harvesting and label curation
  • Learning with weak labels
  • Multimodal learning
  • Image captioning and report generation

The goal of this tutorial is to bridge the gap between medical imaging and medical informatics research, facilitate collaborations between the communities, and introduce new paradigms of multimodal learning exploiting latest innovations across domains.


Yufan Guo (IBM Research)
Mehdi Moradi (IBM Research)
Zhiyong Lu (NLM/NCBI/NIH)



Local arrangement

Chun Lok Wong (IBM Research)