Evaluating the Quality and Diversity of DCGAN-based Generatively Synthesized Diabetic Retinopathy Imagery

Abstract

Publicly available diabetic retinopathy (DR) datasets are imbalanced, containing limited numbers of images with DR. This imbalance contributes to overfitting when training machine learning classifiers. The impact of this imbalance is exacerbated as the severity of the DR stage increases, affecting the classifiers’ diagnostic capacity. The imbalance can be addressed using Generative Adversarial Networks (GANs) to augment the datasets with synthetic images. Generating synthetic images is advantageous if high-quality and diverse images are produced. To evaluate the quality and diversity of synthetic images, several evaluation metrics, such as Multi-Scale Structural Similarity Index (MS-SSIM), Cosine Distance (CD), and Fréchet Inception Distance (FID), are used. Understanding the effectiveness of each metric in evaluating the quality and diversity of synthetic images is critical to select images for augmentation. To date, there has been limited analysis of the appropriateness of these metrics in the context of biomedical imagery. This work contributes an empirical assessment of these evaluation metrics as applied to synthetic Proliferative DR imagery generated by a Deep Convolutional GAN (DCGAN). Furthermore, the metrics’ capacity to indicate the quality and diversity of synthetic images and their correlation with classifier performance are examined. This enables a quantitative selection of synthetic imagery and an informed augmentation strategy, which are often lacking in the literature. Results indicate that FID is suitable for evaluating the quality, while MS-SSIM and CD are suitable for evaluating the diversity of synthetic imagery. Furthermore, the superior performance of Convolutional Neural Network (CNN) and EfficientNet classifiers, as indicated by the and AUC scores, for the augmented datasets compared to the original dataset demonstrate the efficacy of synthetic imagery to augment the imbalanced dataset while improving the classification scores

Paper Google Scholar

Recommended citation: Dragan, Cristina-Madalina and Saad, Muhammad Muneeb and Rehmani, Mubashir Husain and O'Reilly, Ruairi (2023). “Evaluating the Quality and Diversity of DCGAN-based Generatively Synthesized Diabetic Retinopathy Imagery.” Advances in Deep Generative Models for Medical Artificial Intelligence.
Download Paper