advertisement
Abstract #124003 Published in IGR 25-1
Evaluating ChatGPT's Diagnostic Accuracy in Detecting Fundus Images
Gupta A; Al-Kazwini HCureus 2024; 16: e73660
INTRODUCTION: Artificial intelligence is rapidly advancing in healthcare. Ophthalmology, with its reliance on imaging for diagnosis and management, has the potential to benefit from this technology. Deep learning models are currently used in image analysis in ophthalmology. ChatGPT (OpenAI, San Francisco, CA), a large language model, has recently expanded to include image analysis, creating new opportunities for diagnostic applications. While prior research shows potential in text-based diagnostics for ophthalmology, there is limited literature on AI's diagnostic accuracy in interpreting retinal images. METHODS: We selected 12 fundus images from key diseases identified by the Royal College of Ophthalmology curricula for medical students, foundation doctors, and trainees. Each image was presented to ChatGPT 4.0 using a standardised prompt to identify the most likely diagnosis. Responses were recorded, and the model's accuracy was assessed by comparing its diagnoses to the confirmed conditions. RESULTS: ChatGPT accurately diagnosed four out of 12 diseases (papilloedema, dry age-related macular degeneration (ARMD), glaucoma and vitreous haemorrhage) and provided one partially correct diagnosis (diabetic retinopathy). However, the model struggled with seven cases, including central retinal artery occlusion, central retinal vein occlusion, dry ARMD, rhegmatogenous retinal detachment, tractional retinal detachment, epiretinal membrane and macular hole. CONCLUSION: ChatGPT demonstrates the potential for diagnosis of retinal conditions from fundus photography. However, it currently lacks the accuracy required for clinical application; the model often hallucinates when unsure, which has diagnostic implications. Further work is required to refine these models and expand their diagnostic potential.
Ophthalmology, Royal Free Hospital, London, GBR.
Full article