Comparing Technical Failure Rates in Diabetic Retinopathy

Comparing Technical Failure Rates in Diabetic Retinopathy Screening Between RETeval,!
a 30 Hz Flicker Electroretinogram Device, and Mydriatic, 7­ Field, Stereo Fundus Photography!
Maa A1,2, Pillow E3,4,5, Feuer W6,7, Brown T3, Caywood R3, Davis Q8, Chasan J2, Fransen S4,5,9 !
Introduction !
Methods !
•  Subjects were recruited from Veterans Affairs (VA) medical centers in Atlanta, Georgia and
Oklahoma City, Oklahoma. !
•  Patients with diabetes were selectively recruited (Table 2). !
•  Exclusion criteria included subjects with an eye disease other than DR, previous treatment for DR,
or any other eye pathology that, in the opinion of the recruiting ophthalmologist, would result in
ungradable ETDRS images or affect the ERG (e.g. vein occlusion). !
•  The study was conducted in accordance with the Health Insurance Portability and Accountability Act
and applicable FDA regulations. IRB approval was obtained at both institutions and all participants
provided written informed consent.The study is registered on www.clinicaltrials.gov (NCT01950663).!
!
Procedure !
•  The subject’s vision was checked with correction and the RETeval procedure was performed (see
photographs).!
•  The subject evaluated an Amsler grid as an additional method of detecting Clinically Significant
Macular Edema (CSME). !
•  Subjects were dilated with tropicamide and phenylephrine drops and ETDRS, 7 field, color, stereo
fundus photographs were taken.!
!
Analysis !
•  ETDRS images were double-read in a masked fashion in a dedicated reading center (Inoveon
Corporation, Oklahoma City, OK). !
•  Results which differed by more than 1 ETDRS step or with respect to the referral criterion were
adjudicated by the two readers overseen by a retinal specialist. These adjudicated results served as
the gold standard to which the RETeval device results were compared. !
•  Worst eye results were used for both the RETeval device and the gold standard, because subjects
(not eyes) are referred.!
Acknowledgements!
The authors wish to thank the research coordinators, photographers, and readers who helped make this study possible.!
Results !
•  468 subjects started and 467 completed the study (99.8%). No adverse events were reported. !
•  The RETeval device had a technical failure rate (no results generated) of 0.9% (4/467) whereas
ETDRS photography (ungradable images) had a significantly higher (p < 0.001, exact McNemar test)
technical failure rate of 11% (51/467) (Figure 1). The RETeval device generated results on 98%
(50/51) of the subjects who had ungradable images.!
•  RETeval testing time averaged 2.3 minutes to test both eyes (Figure 2). !
•  The RETeval device showed a statistically significant increase in the 30 Hz flicker implicit time as a
function of retinopathy level (Figure 3). !
•  Referable retinopathy (ETDRS 53-85) is significantly higher than the other 3 means (one-way
ANOVA). !
•  The three non-referable retinopathy means are all significantly different from each other (Least
Significant Difference tests). !
•  A more complete data analysis is underway.!
35!
10!
30!
9!
8!
25!
7!
20!
p < 0.001!
6!
%!
5!
!
%!
Discussion !
•  Cataracts and small pupils, even after dilation, cause ungradable images (technical failures) and lead
to potentially unecessary referrals for image based diabetic retinopathy screening systems. !
•  In this study, which had the advantages of dilation, expert photographers, and an exclusion criteria
designed to limit ungradable images, the ETDRS photography technical failure rate was still 11%.!
•  The RETeval device demonstrated a clinically and statistically significantly lower technical failure rate
(Figure 1) because ERG results are unaffected by cataracts and the RETeval device provides results
for pupil sizes as small as 1.3 mm.!
•  Non-mydriatic (undilated) fundus photography is gaining increased acceptance for DR screening to
overcome patient and physician reluctance to dilation. However, the absence of dilation increases the
technical failure rate. Non-mydriatic technical failure rates were reported at 35% in 20065 and 72% in
2013.6 !
•  Some “non-mydriatic” systems dilate 10-20% of their patients to reduce the technical failure rate.!
•  RETeval testing time averaged 2.3 minutes per patient for both eyes, with no dilation or other
preparation required. In comparison, pupil dilation for mydriatic photography takes 20 - 30 minutes.!
•  Training to operate the RETeval device in this trial was less than 1 hour. This compares favorably to
weeks of training required for retinal photographers.!
•  Image based systems require image interpretation by eye care providers or nonphysician expert
graders who are trained to evaluate diabetic retinopathy and typically work in a remote reading
center. This workflow does not provide immediate results to the patient.!
•  The RETeval device provides immediate numerical results which require minimal interpretation.!
•  Reproducibility must be managed in image based systems. In this study, 17% of subjects required
their photographs to be adjudicated due to results being either sufficiently different or differing with
respect to referring the patient to ophthalmology for vision-threatening retinopathy.!
•  Preliminary analysis suggests the RETeval device may be accurate in screening for diabetic
retinopathy.!
15!
4!
Race!
Caucasian!
African American!
American Indian!
Hispanic!
Other!
Asian!
Gender!
Male!
Female!
Diabetes type!
1!
2!
Medication type!
Oral!
Insulin!
Both!
n!
%!
243!
184!
24!
9!
5!
2!
!
409!
58!
!
22!
445!
!
149!
150!
168!
52.0 !
39.0 !
5.1 !
1.9 !
1.1 !
0.4 !
!
87.6 !
12.4 !
!
4.7 !
95.3 !
!
31.9 !
32.1 !
36.0 !
2!
1!
5!
RETeval!
0!
0!
Figure 1. Technical failure rates.!
1!
2!
3!
4!
5!
6!
7!
Figure 2. RETeval testing time (minutes).!
8!
9!
p < 0.001!
p < 0.001!
35!
p < 0.001!
p < 0.001!
!
34!
Conclusions !
p = 0.013!
•  The RETeval device shows promise as a new screening tool for diabetic retinopathy because of its
low technical failure rate, rapid test time, immediate results, ease of use, portability, and lack of
adverse events.!
•  Preliminary analysis suggests the RETeval device may be accurate in screening for diabetic
retinopathy.!
p = 0.038!
Table 2. Disease severity distribution.!
n!
RETeval
% ! Technical
Failures!
93!
20 !
1!
Very mild or mild NPDR (ETDRS 14-35) without CSME! 127!
27 !
0!
Moderate NPDR (ETDRS 43-47) without CSME!
90!
19 !
1!
No NPDR, mild NPDR, or moderate NPDR with CSME!
39!
8!
0!
Servere NPDR (ETDRS 53) or PDR (ETDRS 61-85)!
52!
11 !
1!
CSME with ungradable ETDRS level!
15!
3!
0!
51!
11 !
1!
Severity!
No NPDR (ETDRS 10-12) without CSME!
Ungradable photographs!
10!
3!
Table 1. Subject demographics.!
!
Author Affiliations!
1 Atlanta VA Medical Center, Decatur, GA!
2 Emory University School of Medicine, Atlanta, GA!
3 Oklahoma City VA Medical Center, Oklahoma City, OK!
4 University of Oklahoma College of Medicine, Oklahoma City, OK!
5 Dean McGee Eye Institute, Oklahoma City, OK!
6 University of Miami Miller School of Medicine, Miami, FL!
7 Bascom Palmer Eye Institute, Miami, FL!
8 LKC Technologies, Inc, Gaithersburg, MD!
9 Inoveon Corporation, Oklahoma City, OK!
!
Disclosures!
•  RETeval has not been approved by the FDA.!
•  Supported by Award Number R44EY021121 from the National Eye Institute. The content is solely the responsibility
of the authors and does not necessarily represent the official views of the National Eye Institute of the National
Institutes of Health. Additional funding provided by LKC Technologies, Inc.!
•  Dr. Davis is an employee of LKC Technologies, Inc. The remaining authors receive research support from LKC
Technologies, Inc. for this study.!
ETDRS!Photography!
11!
Total! 467!
4!
32 Td-s Implicit time (ms)!
•  This study compared the diabetic retinopathy (DR) screening performance of a new device,
RETeval™ (LKC Technologies, Inc. Gaithersburg MD), to gold standard Early Treatment Diabetic
Retinopathy Study (ETDRS), 7-field, stereo, color, fundus photography.!
•  DR remains the leading cause of preventable blindness among working age adults in the US.1 !
•  Treatment prevents 90% of this vision loss if threshold disease is detected early.2 !
•  Nonetheless, despite decades of public health awareness campaigns and programs, only 57% of
patients with diabetes received universally recommended eye examinations in 2011.3 !
•  Contributing factors include: !
1)  Primary care physicians do not have the time or equipment to perform adequate retinal
examinations in their office.!
2)  Patients are not evaluated by eye care providers because: a) they are not referred, b) they
do not keep their appointments because of the appointment fatigue of a chronic disease and
they do not want their eyes dilated, c) diabetic retinopathy is asymptomatic until it exceeds
the threshold for treatment.!
3)  Existing image based screening methods require sophisticated equipment and training and
usually require dilation to achieve adequate image quality. !
•  The electroretinogram (ERG) is a test that detects the eye’s electrical response to light.!
•  The ERG has been shown since 1987 to correlate with the severity of DR.4!
•  Existing ERG equipment can not be used to screen for DR because equipment size, long patient
preparation time, requirement for dilation, corneal electrodes, and difficulty interpreting the results, all
make it ineffective in the routine clinical setting.!
•  The RETeval device, a small, simple, fast, hand held, 30Hz flicker ERG device greatly simplifies the
use of ERGs for DR screening by eliminating dilation and using skin sensors (see photographs).!
!
33!
!
References!
1.  Centers for Disease Control. National Diabetes Fact Sheet, 2011. 2011.!
2.  Ferris III FL. How effective are treatments for diabetic retinopathy? JAMA. 1993;269:1290-1291.!
3.  National Committee for Quality Assurance. The State of Health Care Quality 2012. Available at:
http://www.ncqa.org/tabid/836/Default.aspx. Accessed February 17, 2013.!
4.  Bresnick G, Palta M. Temporal Aspects of the Electroretinogram in Diabetic Retinopathy. Arch
Opthalmol. 1987:660-4.!
5.  Ahmed J, Aiello L, Ward T, Cavallerano J, Bursell S, Vigersky R. The Sensitivity and Specificity of
Nonmydriatic Digital Stereoscopic Retinal Imaging in Detecting Diabetic Retinopathy. Diabetes Care.
2006:2205-9!
6.  Invernizzi A, Bevilacqua M, Cozzi M, Bianchi C, Pagani A, Cigada M, Staurenghi G. Telemedicine for
diabetic retinopathy screening: a pilot program for Italian hospitals. Invest Ophthalmol Vis
Sci. 2013:54;E-Abstract 206.!
32!
31!
30!
1365B Clifton Road, NE
Atlanta, GA 30322
29!
10 - 12!
14 - 35!
43 - 47!
53 - 85!
ETDRS Level!
Figure 3. Implicit time versus ETDRS level. Dots represent the mean and the error bars
the standard error of the mean. Arrows show the level of statistical significance of
differences between means.!
www.eyecenter.emory.edu