Clinical diagnosis of the eye is performed over multifarious data modalities including scalar clinical labels, vectorized biomarkers, two-dimensional fundus images, and three-dimensional Optical Coherence Tomography (OCT) scans. While the clinical labels, fundus images and OCT scans are instrumental measurements, the vectorized biomarkers are interpreted attributes from the other measurements. Clinical practitioners use all these data modalities for diagnosing and treating eye diseases like Diabetic Retinopathy (DR) or Diabetic Macular Edema (DME). Enabling usage of machine learning algorithms within the ophthalmic medical domain requires research into the relationships and interactions between these relevant data modalities. Existing datasets are limited in that: (i) they view the problem as disease prediction without assessing biomarkers, and (ii) they do not consider the explicit relationship among all four data modalities over the treatment period. In this paper, we introduce the Ophthalmic Labels for Investigating Visual Eye Semantics (OLIVES) dataset that addresses the above limitations. This is the first OCT and fundus dataset that includes clinical labels, biomarker labels, and time-series patient treatment information from associated clinical trials. The dataset consists of 1268 fundus eye images each with 49 OCT scans, and 16 biomarkers, along with 3 clinical labels and a disease diagnosis of DR or DME. In total, there are 96 eyes’ data averaged over a period of at least two years with each eye treated for an average of 66 weeks and 7 injections. OLIVES dataset has advantages in other fields of machine learning research including self-supervised learning as it provides alternate augmentation schemes that are medically grounded.
Challenges of This Dataset:
While challenges in natural images are generally contrived by intervening on top of data, the complexities in ophthalmic datasets arise because of issues in data collection, inversion, representation and annotation. OLIVES data modalities range from 1 dimensional numerical values (BCVA, Patient ID), vectorized biomarkers, 2-dimensional fundus images, and 3-dimensional scans (optical coherence tomography). Moreover, some of this data is objectively measured through instruments from patients (fundus, OCT), subjectively collected through eye tests (BCVA), while other data is interpreted and openly adjudicated through images (biomarkers). The variation within scans between visits can be minimal while the difference in manifestation of the same disease between patients may be substantial. This is shown in Fig. 1. The domain difference between OCT scans can arise due to pathology manifestation between patients (Fig. 1a and Fig. 1b), clinical labels (Fig. 1c), and the visit along the treatment process when the scan is taken (Fig. 1d). OLIVES provides access to these challenging data modalities that allow for innovative ML algorithms.
Access Code and Dataset
Images and Labels: https://doi.org/10.5281/zenodo.7105232
Code: https://github.com/olivesgatech/OLIVES_Dataset?tab=readme-ov-file
Preprint: https://arxiv.org/abs/2209.11195
Update (2/10/2025): The dataset (as separate disease classification and biomarker detection splits) is available on HuggingFace: https://huggingface.co/datasets/gOLIVES/OLIVES_Dataset
Citations
If you find the work useful, please include the following citation in your work:
@inproceedings{prabhushankarolives2022, title={OLIVES Dataset: Ophthalmic Labels for Investigating Visual Eye Semantics}, author={Prabhushankar, Mohit and Kokilepersaud, Kiran and Logan, Yash-yee and Trejo Corona, Stephanie and AlRegib, Ghassan and Wykoff, Charles}, booktitle={Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 2 (NeurIPS Datasets and Benchmarks 2022) }, year={2022} }
