Home  View Data  Annotate Data  Register  Download  Submit  Results  FAQ  Programs 

Retinopathy Online Challenge

Welcome to the Retinopathy Online Challenge (ROC) website. The ROC aims to help patients with diabetes through improving computer aided detection and diagnosis (CAD) of diabetic retinopathy. Diabetic retinopathy is the second largest cause of blindness in the US and Europe. Most visual loss and blindness from diabetic retinopathy can be prevented through early diagnosis or screening.

Computer algorithms have been developed in order to detect the signs of diabetic retinopathy from retinal images obtained using a digital retinal camera. However, few, if any of these algorithms have been applied in clinical practice. ROC facilitates the translation of diabetic retinopathy CAD into clinical practice by:

  • enabling any medical image analysis research group to develop diabetic retinopathy CAD algorithms by offering a training set of retinal images with reference standard provided by internationally accepted retinal experts.
  • evaluating the output of a diabetic retinopathy CAD algorithm in a uniform manner on a supplied test set, allowing algorithms to be compared both to other algorithms and retinal experts.
  • organizing meetings and workshops at international conferences to compare CAD systems, following the paradigm of revolution through competition.

    • Currently, we have released a first data set, aimed at CAD of microaneurysms and dot hemorrhages. These abnormalities are amongst the first signs of the presence of diabetic retinopathy. For some example lesions see Figure 1 below. On this site, interested research groups and companies can register a team, download the data and submit results (multiple submissions possible). The results consist of a set of image-locations combined with a 'degree of suspicion' for each location that it is an abnormality. Each of the submitted results will be analyzed in the same way using FROC analysis. Based on this each submission will be awarded a score.

    The first teams have submitted their system's results to the competition website, they were presented at the CAD Conference of SPIE: Medical Imaging 2009. The results were presented as regular conference contributions and in a special workshop included in that meeting. An overview paper describing the results of the competition, jointly authored by all teams, will be submitted to a high ranking journal. New teams can still sign up and participate in the competition although their results will not be included in the overview paper. The ROC micoaneurysm dataset will remain available online via this website.

    In addition to this competition, powerful online annotation tools available on this site enable experts to browse image data on-line, annotate it, and in the future test their reading skills and compare their detection performance with the reference standard and with that of CAD.



    Figure 1: Details from retinal images with lesion markers.

    The content of this website is copyright 2007 (c) The University of Iowa and the ROC organizers (Michael D. Abramoff, Bram van Ginneken and Meindert Niemeijer). All rights are reserved. Permission is granted to others to use the ROC image data for developing and testing of diabetic retinopathy algorithms purposes only as described in the competition rules. Any use of the datasets, although free, must reference the ROC (see competition rules). The datasets may not be changed in any way, nor may derivate works be produced without express written permission. No commercial permission is granted without a separate written license from the University of Iowa. Funded in part by the National Eye Institute (5R01EY017066), Research to Prevent Blindness, NY, NY and the Netherlands Organization for Health Related Research.