Most current diagnostic devices are expensive, require trained specialists to operate and gather static images with sparse data points. This leads to preventable diseases going undetected until late stage, resulting in greatly narrowed treatment options. This is especially true for retinal imaging. Future solutions are low cost, portable, self-administered by the patient, and capable of providing multiple data points, population analysis, and trending. This enables preventative interventions through mass accessibility, constant monitoring, and predictive modeling. Retinal imaging potential Retinal imaging offers a unique perspective into the health of our eyes as well as that of our body. Ocular-specific diseases and life-threatening systemic diseases show manifestations in the eye, including age-related macular degeneration (AMD) and glaucoma, two of the leading causes of blindness, as well as diabetes, hypertension, and multiple sclerosis (MS). Additionally, an individual's lifestyle and environmental factors are reflected in the retinal vasculature. 1,2 Disease detection is oftentimes about identifying changes in the structure of the blood vessels, timing the changes and locating them. Characteristic structural changes can be distinguished by examining different features (optic fundus signs): flame-shaped hemorrhages (nerve fiber layer), vitreous (cloudy) hemorrhages, preretinal (between vitreous membrane and retina) hemorrhages; venous beading (dilation of the vessel walls); hard exudates (lipid deposits) and soft exudates (microinfarction of retinal nerve fiber layer); microaneurysms (MAs); and drusen. 3 Knowing when a feature occurred is key. For example, the MA population is dynamic and changes occur in a matter of months. 4,5 For diabetic retinopathy (DR), it has been established that MAs are the earliest lesions visible. 6 Additionally, MA turnover rates are indicative of early-stage DR as well as the likelihood of DR progression to macular edema. 7 Even more critical is the location of the features, ie, the particular site where blood vessel changes occur. Changes affecting the area of central vision (fovea/ macula) have the most severe effects on vision, eg, edematous changes (swelling, fluid accumulation). 8 Similar changes occurring in the peripheral retina often go unnoticed by the subject. Indeed, many vision disorders start at the periphery and later spread to the center of the retina. 9 Therefore vision loss is often considered a "silent" disease, only noticed by patients at late stages. To be able to prevent disease progression, it is critical to detect such changes early on. In addition, retinal vessel patterns can Correspondence: ramesh raskar Massachusetts institute of technology,

Many important eye diseases as well as systemic diseases manifest themselves in the retina. While a number of other anatomical structures contribute to the process of vision, this review focuses on retinal imaging and image analysis. Following a brief overview of the most prevalent causes of blindness in the industrialized world that includes age-related macular degener-ation, diabetic retinopathy, and glaucoma, the review is devoted to retinal imaging and image analysis methods and their clinical implications. Methods for 2-D fundus imaging and techniques for 3-D optical coherence tomography (OCT) imaging are reviewed. Special attention is given to quantitative techniques for analysis of fundus photographs with a focus on clinically relevant assessment of retinal vasculature, identification of retinal lesions, assessment of optic nerve head (ONH) shape, building retinal atlases, and to automated methods for population screening for retinal diseases. A separate section is devoted to 3-D analysis of OCT images, describing methods for segmentation and analysis of retinal layers, retinal vasculature, and 2-D/3-D detection of symptomatic exu-date-associated derangements, as well as to OCT-based analysis of ONH morphology and shape. Throughout the paper, aspects of image acquisition, image analysis, and clinical relevance are treated together considering their mutually interlinked relationships .

Diabetes is a chronic end organ disease that occurs when the pancreas does not secrete enough insulin or the body is unable to process it properly. Over time, diabetes affects the circulatory system, including that of the retina. Diabetic retinopathy is a medical condition where the retina is damaged because fluid leaks from blood vessels into the retina. Ophthalmologists recognize diabetic retinopathy based on features, such as blood vessel area, exudes, hemorrhages, microaneurysms and texture. In this paper we review algorithms used for the extraction of these features from digital fundus images. Furthermore, we discuss systems that use these features to classify individual fundus images. The classifications efficiency of different DR systems is discussed. Most of the reported systems are highly optimized with respect to the analyzed fundus images, therefore a generalization of individual results is difficult. However, this review shows that the classification results improved has improved recently, and it is getting closer to the classification capabilities of human ophthalmologists.

Graphical Abstract Highlights d An artificial intelligence system using transfer learning techniques was developed d It effectively classified images for macular degeneration and diabetic retinopathy d It also accurately distinguished bacterial and viral pneumonia on chest X-rays d This has potential for generalized high-impact application in biomedical imaging In Brief Image-based deep learning classifies macular degeneration and diabetic retinopathy using retinal optical coherence tomography images and has potential for generalized applications in biomedical image interpretation and medical decision making. SUMMARY The implementation of clinical-decision support algorithms for medical imaging faces challenges with reliability and interpretability. Here, we establish a diagnostic tool based on a deep-learning framework for the screening of patients with common treatable blinding retinal diseases. Our framework utilizes transfer learning, which trains a neural network with a fraction of the data of conventional approaches. Applying this approach to a dataset of optical coherence tomography images, we demonstrate performance comparable to that of human experts in classifying age-related macular degeneration and diabetic macu-lar edema. We also provide a more transparent and interpretable diagnosis by highlighting the regions recognized by the neural network. We further demonstrate the general applicability of our AI system for diagnosis of pediatric pneumonia using chest X-ray images. This tool may ultimately aid in expediting the diagnosis and referral of these treatable conditions, thereby facilitating earlier treatment, resulting in improved clinical outcomes.

Purpose: To determine if the area of the foveal avascular zone (FAZ) is correlated with visual acuity (VA) in diabetic retinopathy (DR) and retinal vein occlusion (RVO). Design: Cross-sectional study. Participants: Ninety-five eyes of 66 subjects with DR (65 eyes), branch retinal vein occlusion (19 eyes), and central retinal vein occlusion (11 eyes). Methods: Structural optical coherence tomography (OCT; Spectralis, Heidelberg Engineering) and OCT angiography (OCTA; Avanti, Optovue RTVue XR) data from a single visit were analyzed. FAZ area, point thickness of central fovea, central 1-mm subfield thickness, the occurrence of intraretinal cysts, ellipsoid zone disruption, and disorganization of retinal inner layers (DRIL) length were measured. VA was also recorded. Correlations between FAZ area and VA were explored using regression models. Main outcome measure was VA. Results: Mean age was 62.9AE13.2 years. There was no difference in demographic and OCT-derived anatomic measurements between branch retinal vein occlusion and central retinal vein occlusion groups (all P ! 0.058); therefore, data from the 2 groups were pooled together to a single RVO group for further statistical comparisons. Univariate and multiple regression analysis showed that the area of the FAZ was significantly correlated with VA in DR and RVO (all P 0.003). The relationship between FAZ area and VA varied with age (P ¼ 0.026) such that for a constant FAZ area, an increase in patient age was associated with poorer vision (rise in logarithm of the minimum angle of resolution visual acuity). Disruption of the ellipsoid zone was significantly correlated with VA in univariate and multiple regression analysis (both P < 0.001). Occurrence of intraretinal cysts, DRIL length, and lens status were significantly correlated with VA in the univariate regression analysis (P 0.018) but not the multiple regression analysis (P ! 0.210). Remaining variables evaluated in this study were not predictive of VA (all P ! 0.225). Conclusions: The area of the FAZ is significantly correlated with VA in DR and RVO and this relationship is modulated by patient age. Further study about FAZ area and VA correlations during the natural course of retinal vascular diseases and following treatment is warranted. Ophthalmology 2016;123:2352-2367 ª 2016 by the American Academy of Ophthalmology. Supplemental material is available at www.aaojournal.org. Diabetic retinopathy (DR) is the leading cause of vision loss in the working-age population in developed countries. 1 According to data from the National Eye Institute, there were approximately 7.7 million diagnosed cases of DR in the United States alone in 2010. 2 Retinal venous occlusion (RVO), due to central retinal vein occlusion (CRVO) or branch retinal vein occlusion (BRVO), is the second most common retinal vascular disorder leading to significant vision loss, with an estimated number of global cases in 2010 exceeding 16.4 million. 3 The prevalence of DR and RVO

视网膜疾病的自动临床诊断已成为一种有前途的方法，可以促进在专家访问受限的地区发现。基于眼底结构和血管疾病是视网膜疾病的主要特征这一事实，我们提出了一种新的视觉辅助诊断混合模型。支持向量机（SVM）和深度神经网络（DNN）。此外，我们提出了一种新的临床视网膜数据集，名为EyeNet2，用于包含52种视网膜疾病类别的眼科学。使用EyeNet2，我们的模型诊断准确率达到90.43％，模型性能可与专业眼科医生相媲美。

M edical imaging is expanding globally at an unprecedented rate 1,2 , leading to an ever-expanding quantity of data that requires human expertise and judgement to interpret and triage. In many clinical specialities there is a relative shortage of this expertise to provide timely diagnosis and referral. For example, in ophthalmology, the widespread availability of optical coherence tomography (OCT) has not been matched by the availability of expert humans to interpret scans and refer patients to the appropriate clinical care 3. This problem is exacerbated by the marked increase in prevalence of sight-threatening diseases for which OCT is the gold standard of initial assessment 4-7. Artificial intelligence (AI) provides a promising solution for such medical image interpretation and triage, but despite recent breakthrough studies in which expert-level performance on two-dimensional photographs in preclinical settings has been demonstrated 8,9 , prospective clinical application of this technology remains stymied by three key challenges. First, AI (typically trained on hundreds of thousands of examples from one canonical dataset) must generalize to new populations and devices without a substantial loss of performance , and without prohibitive data requirements for retraining. Second, AI tools must be applicable to real-world scans, problems and pathways, and designed for clinical evaluation and deployment. Finally, AI tools must match or exceed the performance of human experts in such real-world situations. Recent work applying AI to OCT has shown promise in resolving some of these criteria in isolation , but has not yet shown clinical applicability by resolving all three.

年龄相关性黄斑变性（AMD）是一种无症状的视网膜疾病，可能导致视力丧失。获得高质量的相关视网膜图像的机会有限，并且对定义该疾病的子类的特征缺乏了解。在机器学习的最新进展的推动下，我们使用生成对抗网络（GAN）和风格转移专门探索生成建模的潜力，通过特征提取促进临床诊断和疾病理解。我们设计了分析管道，首先从临床图像中生成合成的视网膜图像;应用后续验证步骤。在合成步骤中，我们将GAN（DCGAN和WGAN架构）和样式转移用于图像生成，而验证的步骤控制生成图像的准确性。我们发现生成的图像包含足够的病理学细节，以促进眼科医生的疾病分类任务和疾病相关特征的发现。特别地，我们的系统预测AMD的玻璃疣和地理萎缩子类。此外，使用CAN图像用于GAN的性能优于仅使用原始临床数据集的分类。我们的结果使用现有的视网膜疾病分类器和类激活图进行评估，支持合成图像的预测能力及其用于特征提取的效用。我们的代码示例可在线获取。

据我们所知，我们报告了生成对抗网络（GAN）的第一次端到端应用，用于合成视网膜的光学相干断层扫描（OCT）图像。在给定数据类型的采样时，生成模型已经获得了他们可以合成的越来越逼真的图像的最近的注意力。在本文中，我们将GAN应用于视网膜OCT的采样分布。我们观察到现实OCT图像的合成，描绘了可识别的病理学，例如黄斑裂孔，脉络膜血管膜，近视变性，黄斑囊样水肿和中心性视网膜病变等。这是其第一次这样的报道。这项新技术的潜在应用包括手术模拟，治疗计划，疾病预测，以及加速开发新药和治疗视网膜疾病的外科手术。

Objective: To define quantitative indicators for the presence of intermediate age-related macular degener-ation (AMD) via spectral-domain optical coherence tomography (SD-OCT) imaging of older adults. Design: Evaluation of diagnostic test and technology. Participants and Controls: One eye from 115 elderly subjects without AMD and 269 subjects with intermediate AMD from the Age-Related Eye Disease Study 2 (AREDS2) Ancillary SD-OCT Study. Methods: We semiautomatically delineated the retinal pigment epithelium (RPE) and RPE drusen complex (RPEDC, the axial distance from the apex of the drusen and RPE layer to Bruch's membrane) and total retina (TR, the axial distance between the inner limiting and Bruch's membranes) boundaries. We registered and averaged the thickness maps from control subjects to generate a map of "normal" non-AMD thickness. We considered RPEDC thicknesses larger or smaller than 3 standard deviations from the mean as abnormal, indicating drusen or geographic atrophy (GA), respectively. We measured TR volumes, RPEDC volumes, and abnormal RPEDC thickening and thinning volumes for each subject. By using different combinations of these 4 disease indicators, we designed 5 automated classifiers for the presence of AMD on the basis of the generalized linear model regression framework. We trained and evaluated the performance of these classifiers using the leave-one-out method. Main Outcome Measures: The range and topographic distribution of the RPEDC and TR thicknesses in a 5-mm diameter cylinder centered at the fovea. Results: The most efficient method for separating AMD and control eyes required all 4 disease indicators. The area under the curve (AUC) of the receiver operating characteristic (ROC) for this classifier was >0.99. Overall neurosensory retinal thickening in eyes with AMD versus control eyes in our study contrasts with previous smaller studies. Conclusions: We identified and validated efficient biometrics to distinguish AMD from normal eyes by analyzing the topographic distribution of normal and abnormal RPEDC thicknesses across a large atlas of eyes. We created an online atlas to share the 38 400 SD-OCT images in this study, their corresponding segmentations, and quantitative measurements.

视网膜疾病的自动临床诊断已经成为一种有前途的方法，可以在专家访问受限的地区进行发现。我们提出了一种基于支持向量机（SVM）和深度神经网络（DNN）的新型视觉辅助诊断混合模型。该模型结合了DNN和SVM的互补优势。此外，我们提出了一个新的临床标签集合，用于眼科学，包括32个视网膜疾病类别。使用EyeNet，我们的模型诊断准确率达到89.73％，模型性能可与专业眼科医生相媲美。

Retinal fundus image is an important image modality to document the health of the retina and is widely used to diagnose ocular diseases such as glaucoma, diabetic retinopathy and age-related macular degeneration. However, the enormous amount of retinal data obtained nowadays mostly remains locally; the valuable clinical knowledge is not efficiently exploited. In this study we present an online depository, ORIGA-light , which aims to share clinical ground-truth retinal images with the public, providing open access for researchers to benchmark their computer-aided segmentation systems. We developed an in-house image segmentation and grading tool to facilitate the construction of ORIGA-light. We proposed a quantified assessment method for objective benchmarking, focusing on optic disc and cup segmentation and Cup-to-Disc Ratio (CDR). Currently, ORIGA-light contains 650 retinal images annotated by trained professionals from Singapore Eye Research Institute. The images cover a wide collection of image signs which are critical for glaucoma diagnosis. We will update the system continuously with more clinical ground-truth images. ORIGA-light is available for public access upon request.

在这项工作中，我们提出了一个先进的基于AI的OCT图像分级系统。该系统是一个非常深入的完全卷积的注意力分类网络，通过端到端的高级转移学习和在线randomaugmentation进行训练。它使用准随机增强技术，在推理过程中输出疾病患病率的置信度值。它是一个全自动视网膜OCT分析AI系统，无需任何离线预处理/后处理步骤或手动特征提取，能够理解病理性病变。我们在公开的Mendeley OCTdataset上展示了最先进的性能。

Deep learning emerges as a powerful tool for analyzing medical images. Retinal disease detection by using computer-aided diagnosis from fundus image has emerged as a new method. We applied deep learning convolutional neural network by using MatConvNet for an automated detection of multiple retinal diseases with fundus photographs involved in STructured Analysis of the REtina (STARE) database. Dataset was built by expanding data on 10 categories, including normal retina and nine retinal diseases. The optimal outcomes were acquired by using a random forest transfer learning based on VGG-19 architecture. The classification results depended greatly on the number of categories. As the number of categories increased, the performance of deep learning models was diminished. When all 10 categories were included, we obtained results with an accuracy of 30.5%, relative classi-fier information (RCI) of 0.052, and Cohen's kappa of 0.224. Considering three integrated normal, background diabetic retinopathy, and dry age-related macular degeneration, the multi-categorical classifier showed accuracy of 72.8%, 0.283 RCI, and 0.577 kappa. In addition , several ensemble classifiers enhanced the multi-categorical classification performance. The transfer learning incorporated with ensemble classifier of clustering and voting approach presented the best performance with accuracy of 36.7%, 0.053 RCI, and 0.225 kappa in the 10 retinal diseases classification problem. First, due to the small size of data-sets, the deep learning techniques in this study were ineffective to be applied in clinics where numerous patients suffering from various types of retinal disorders visit for diagnosis and treatment. Second, we found that the transfer learning incorporated with ensemble clas-sifiers can improve the classification performance in order to detect multi-categorical retinal diseases. Further studies should confirm the effectiveness of algorithms with large datasets obtained from hospitals. PLOS ONE | https://doi.org/10.1371/journal.pone.

光学相干断层扫描（OCT）已成为眼科学中最重要的成像模式。最近，大量研究被用于开发机器学习（ML）模型，用于OCT图像中病理特征的识别和量化。在ML模型必须处理的几个可变性来源中，主要因素是采集设备，其可以限制ML模型的可归一化性。在本文中，我们建议通过使用不受监督的非配对图像变换算法CycleGAN来降低不同OCT设备（Spectralis和Cirrus）的图像变化。在视网膜液分割的设置中评估该方法的有用性，即视网膜下囊液（IRC）和视网膜下液（SRF）。首先，我们在使用源OCT设备获取的图像上训练分段模型。然后我们在（1）源，（2）目标和（3）目标OCT图像的变换版本上评估模型。所提出的转换策略显示IRC（SRF）分割的F1得分为0.4（0.51）。与传统的转换方法相比，这意味着F1得分为0.2（0.12）。

This paper proposes a method for automatic classification of spectral domain OCT data for the identification of patients with retinal diseases such as Diabetic Macular Edema (DME). We address this issue as an anomaly detection problem and propose a method that not only allows the classification of the OCT volume, but also allows the identification of the individual diseased B-scans inside the volume. Our approach is based on modeling the appearance of normal OCT images with a Gaussian Mixture Model (GMM) and detecting abnormal OCT images as outliers. The classification of an OCT volume is based on the number of detected outliers. Experimental results with two different datasets show that the proposed method achieves a sensitivity and a specificity of 80% and 93% on the first dataset, and 100% and 80% on the second one. Moreover, the experiments show that the proposed method achieves better classification performance than other recently published works.

使用深度学习算法对眼底照片上的青光眼损伤进行分类的先前方法受到对参考训练集的人类标记的要求的限制。我们提出了一种新的方法，使用光谱域光学相干断层扫描（SDOCT）数据来训练深度学习算法，以量化视觉光学照片上的青光眼结构损伤。该数据集包括来自1,198名受试者的2,312只眼睛的32,820对视盘照片和SDOCTretinal神经纤维层（RNFL）扫描。训练了Adeep学习卷积神经网络以评估视觉光学照片并预测SDOCT平均RNFL厚度。在独立的测试样本中评估算法的性能。测试组中所有6,292个视盘光照片的平均RNFL厚度平均预测值为83.3 $ \ pm $ 14.5 $ \ mu $ m，而所有相应的SDOCT扫描的平均RNFL厚度为82.5 $ \ pm $ 16.8 $ \ mu $ m （P = 0.164）。预测和观察到的RNFL厚度值之间存在非常强的相关性（r = 0.832; P <0.001），预测值的平均绝对误差为7.39 $μM。接受者操作特征曲线区域区分健康眼睛的青光眼深度学习预测和实际SDOCT测量分别为0.944（95 $ \％$ CI：0.912-0.966）和0.940（95 $ \％$ CI：0.902 -0.966）（P = 0.724）。总之，我们引入了一种新颖的深度学习方法来评估视盘照片并提供有关神经损伤量的定量信息。这种方法可能用于诊断和治疗视盘照片的青光眼损伤。

深度学习在各种计算机视觉任务中的出色表现激发了其在医学图像分析中的应用，特别是视网膜神经图像分析。它已被应用于各种任务，包括视网膜眼底图像病理的诊断，检测和分割。已经提出了许多基于深度学习的技术来分析视网膜神经图像，用于黄斑变性和糖尿病视网膜病变的自动检测和诊断。糖尿病视网膜病变的自动检测具有通过促进糖尿病患者的检查来预防视力丧失和失明的潜力。我们对最深层学习技术及其在眼底图像分析中的应用进行了全面研究。本文介绍了与糖尿病视网膜病变图像分析相关的深度学习的关键概念，并回顾了该领域最新的基于深度学习的贡献。我们总结了该论文，总结了最新技术，对未来研究的开放性挑战和方向的批判性讨论。

糖尿病视网膜病变（DR）是一种不断恶化的疾病，是视力障碍和失明的主要原因之一。不同等级之间的细微区别以及许多重要小特征的存在使得识别任务非常具有挑战性。另外，本发明的视网膜病变检测方法是非常费力且耗时的任务，其严重依赖于医生的技能。自动检测糖尿病性视网膜病变对于解决这些问题至关重要。糖尿病视网膜病变的早期检测对于诊断也是非常重要的，可以通过适当的治疗来预防失明。在本文中，我们开发了一种新的深度卷积神经网络，通过识别所有微动脉瘤（MA），DR的最初迹象，以及正确地将标签分配给视网膜眼底图像进行早期检测，视网膜眼底图像被分为五个类别。我们在最大的公众可用的Kagglediabetic视网膜病变数据集上测试了我们的网络，并获得了0.851二次加权kappa评分和0.844 AUC评分，从而实现了最先进的严重性评分。在早期检测中，我们已经实现了98％的灵敏度和94％以上的特异性，这证明了我们提出的方法的有效性。我们提出的架构同时在计算时间和空间方面非常简单和有效。

医学图像中标记物的识别和定量对于诊断，预后和疾病管理是关键的。有监督的机器学习能够检测和利用专家对训练样例进行注释后已知的发现。然而，由于必要的训练样例的数量以及标记词汇对已知实体的限制，监督不能很好地扩展。在这个概念验证研究中，我们提出无监督识别异常作为视网膜光学相干断层扫描（OCT）成像数据中的标记的候选者，而不对先验定义进行约束。我们对数据中经常出现的标记进行识别和分类，并证明这些标记在检测疾病的任务中具有预测价值。对已鉴定的数据驱动标记物进行仔细的定性分析，揭示了它们的可量化发生率与我们目前对疾病进程的理解，早期和晚期相关性黄斑变性（AMD）患者的一致性。在健康图像上训练多尺度深度自动编码器，并且一类支持向量机识别新数据中的异常。异常中的聚类识别稳定的类别。使用这些标记对健康，早期AMD和晚期AMD病例进行分类可获得81.40％的准确度。在公开可用数据集（健康与中间AMD）的第二次二元分类实验中，该模型实现了ROC曲线下面积为0.944的区域。