S7595 - Building Truly Large-Scale Medical Image Databases: Deep Label Discovery and Open-Ended Recognition

The recent rapid and tremendous success of deep neural networks on many challenging computer vision tasks derives from the accessibility of the well-annotated ImageNet and PASCAL VOC datasets. Nevertheless, unsupervised image categorization (that is, without ground-truth labeling) is much less investigated, critically important, and difficult when annotations are extremely hard to obtain in the conventional way of "Google Search" + crowd sourcing (exactly how ImageNet was constructed). We'll present recent work on building two truly large-scale radiology image databases at NIH to boost the development in this important domain. The first one is a chest X-ray database of 110,000+ images from 30,000+ patients, where the image labels were obtained by sophisticated natural language processing-based text mining and the image recognition benchmarks were conducted using weakly supervised deep learning. The other database contains about 216,000 CT/MRI images with key medical findings from 61,845 unique patients, where a new looped deep pseudo-task optimization framework is proposed for joint mining of deep CNN features and image labels. Both medical image databases will be released to the public