IOVS
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
 QUICK SEARCH:   [advanced]


     


This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when eLetters are posted
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Bowd, C.
Right arrow Articles by Weinreb, R. N.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Bowd, C.
Right arrow Articles by Weinreb, R. N.
(Investigative Ophthalmology and Visual Science. 2002;43:3444-3454.)
© 2002 by The Association for Research in Vision and Ophthalmology, Inc.

Comparing Neural Networks and Linear Discriminant Functions for Glaucoma Detection Using Confocal Scanning Laser Ophthalmoscopy of the Optic Disc

Christopher Bowd1, Kwokleung Chan2,3, Linda M. Zangwill1, Michael H. Goldbaum1, Te-Won Lee2,3, Terrence J. Sejnowski2,3 and Robert N. Weinreb1

1 From the Hamilton Glaucoma Center and the 2 Institute for Neural Computation, University of California, San Diego, La Jolla, California; and the 3 Computational Neurobiology Laboratories, The Salk Institute, La Jolla, California.


    Abstract
 Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 References
 
PURPOSE. To determine whether neural network techniques can improve differentiation between glaucomatous and nonglaucomatous eyes, using the optic disc topography parameters of the Heidelberg Retina Tomograph (HRT; Heidelberg Engineering, Heidelberg, Germany).

METHODS. With the HRT, one eye was imaged from each of 108 patients with glaucoma (defined as having repeatable visual field defects with standard automated perimetry) and 189 subjects without glaucoma (no visual field defects with healthy-appearing optic disc and retinal nerve fiber layer on clinical examination) and the optic nerve topography was defined by 17 global and 66 regional HRT parameters. With all the HRT parameters used as input, receiver operating characteristic (ROC) curves were generated for the classification of eyes, by three neural network techniques: linear and Gaussian support vector machines (SVM linear and SVM Gaussian, respectively) and a multilayer perceptron (MLP), as well as four previously proposed linear discriminant functions (LDFs) and one LDF developed on the current data with all HRT parameters used as input.

RESULTS. The areas under the ROC curves for SVM linear and SVM Gaussian were 0.938 and 0.945, respectively; for MLP, 0.941; for the current LDF, 0.906; and for the best previously proposed LDF, 0.890. With the use of forward selection and backward elimination optimization techniques, the areas under the ROC curves for SVM Gaussian and the current LDF were increased to approximately 0.96.

CONCLUSIONS. Trained neural networks, with global and regional HRT parameters used as input, improve on previously proposed HRT parameter-based LDFs for discriminating between glaucomatous and nonglaucomatous eyes. The performance of both neural networks and LDFs can be improved with optimization of the features in the input. Neural network analyses show promise for increasing diagnostic accuracy of tests for glaucoma.


    Introduction
 Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 References
 
Methods of early detection of glaucoma often focus on the assessment of optic disc topography and retinal nerve fiber layer (RNFL) thickness in an attempt to identify patients at risk for development of visual field defects. Because clinical examination and fundus photography are subjective and qualitative, optical imaging techniques that provide objective and quantitative measures for evaluating the optic disc and RNFL may be advantageous. For example, confocal scanning laser ophthalmoscopy (CSLO) provides quantitative measures that are reproducible and correlate with histomorphometric measurements in monkey eyes.1 2 3 4 5 CSLO shows promise for discriminating between eyes with characteristic glaucomatous damage and healthy eyes, although the reported success for classifying these eyes varies.6 7 8 9

In an attempt to classify eyes effectively as glaucomatous or healthy, analysis strategies have been developed that use as input different CSLO optic disc topography measurement parameters, by using statistical methods such as linear discriminant function (LDF) analyses.8 10 11 12 13 14 LDF analysis assumes that data representing different groups are linearly separable. If this assumption is not well met, the classifier’s performance is degraded. Other investigators have used artificial neural networks (specifically, multilayer perceptrons [MLPs] with back-propagated learning) trained on CSLO parameters to classify eyes as glaucomatous or healthy.8 15 Using this method, the neural network classifier is trained to detect a relationship between input (CSLO parameters) and a predefined gold-standard diagnosis by comparing its prediction with the labeled diagnosis and by learning from its mistakes. In general, neural network techniques differ from basic statistical techniques such as LDFs, because they can adapt to the distribution of the data rather than assume a predefined distribution. The success of statistical or neural network classification methods is most often measured by reporting areas under the receiver operating characteristic (ROC) curve or by reporting sensitivity at different specificities.

The purpose of the current study was to compare the performance of previously proposed HRT parameter-based LDFs with three artificial neural network methods in a single sample. Comparing different classification methods in a single sample reduces the effects of confounding variables, such as subject demographics and severity of glaucoma. Because of their adaptability, we hypothesized that neural network techniques would perform as well as or better than LDF classifiers in discriminating between glaucomatous and healthy eyes.


    Methods
 Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 References
 
Subjects
One randomly selected eye from each of 108 patients with glaucoma and 189 normal subjects was included in the study. All subjects underwent a complete ophthalmic examination, including slit lamp biomicroscopy, measurement of intraocular pressure (IOP), stereoscopic fundus examination, stereoscopic photography of the optic disc, and standard full-threshold automated perimetry (SAP; Humphrey Field Analyzer, Humphrey Instruments, Dublin, CA). Informed consent was obtained from all participants and the University of California, San Diego Human Subjects Committee approved all methodology. All methods adhered to the provisions of the Declaration of Helsinki guidelines for research in human subjects.

Because CSLO-measured optic disc topography was being evaluated, we chose the best indicator of glaucoma that is not dependent on optic disc appearance for training and evaluating the neural network techniques. Patients with open-angle glaucoma were defined as those with at least two consecutive SAP fields with either a corrected pattern standard deviation (CPSD) outside the 95% normal limits or a glaucoma hemifield test (GHT) result outside the 99% normal limits. At least one of the abnormal fields was obtained on or before the date of CSLO imaging. Mean deviation (±SD) of the SAP closest to the CSLO imaging date was -6.08 ± 5.77 dB, indicating mild to moderate visual field damage. Patients with glaucoma had no history of diabetes and no apparent cataracts and were not using medication known to affect visual sensitivity at the time of visual field testing. Best corrected visual acuity at the time of SAP and CSLO testing was 20/40 or better. The average age (±SD) of patients with glaucoma was 65.2 ± 13.6 years.

Healthy eyes had a measured IOP of 22 mm Hg or more with no history of elevated IOP. These eyes had intact rims, no evidence of hemorrhage, notching, glaucomatous excavation, or RNFL defect and had symmetrical optic discs (asymmetry of vertical cup-disc ratio < 0.2) based on clinical examination. SAP results were within normal limits. Healthy patients had no history of diabetes or other systemic disease and no ophthalmic or neurologic surgery or disease. Best corrected visual acuity at the time of testing was 20/40 or better. The average age (±SD) of healthy subjects was 54.2 ± 16.3 years, significantly younger than patients with glaucoma (t-test; P < 0.05).

Confocal Scanning Laser Ophthalmoscope
The Heidelberg Retina Tomograph (HRT-1, Heidelberg Engineering, Heidelberg, Germany) provides topographical measures of the optic disc and parapapillary retina, with confocal scanning laser technology. The topographical image is derived from 32 optical sections at consecutive focal depth planes. Each image consists of 256 x 256 pixels with each pixel corresponding to retinal height at its location. This instrument has been discussed in detail elsewhere.1 16

Procedure.
Three 15° field-of-view scans centered on the optic disc and judged to be of acceptable quality were obtained for each test eye. A mean topography image of these three scans was created with the HRT. The optic disc margin was outlined on the mean topography image by a trained technician using information obtained by viewing stereoscopic photographs of the optic disc.

HRT Parameters.
Eighty-three topographic parameters (automatically provided by HRT software, ver. 2.01) were used in this study (Table 1) . We used global (360-degree) measures for each parameter and for some parameters, also used regional measures. Regions were defined as temporal superior (46–90° unit circle), nasal superior (91–135°), nasal (136–225°), nasal inferior (226–270°), temporal inferior (271–315°), and temporal (316–45°). Regional parameters were not evaluated for height variation of contour, mean cup depth, RNFL thickness, RNFL cross-sectional area, reference height, or rim area. All these parameters have been discussed in more detail elsewhere.1 16 17 18


View this table:
[in this window]
[in a new window]
 
Table 1. HRT Parameters Included in the Full Dimensional Input Set

 
Linear Discriminant Functions
We evaluated the performance of four published linear discriminant analysis formulas developed by Mikelberg et al.,10 Bathija et al.,12 Mardin et al.,11 and Iester et al.14 (and Iester M, personal communication, June 2001) for classifying eyes as glaucomatous or healthy. These formulas were developed with the available optic disc topography parameters provided by the HRT software. The Mikelberg et al.10 formula is available in HRT software version 2.01 as "glaucoma classification."
LDF Mikelberg et al.10 : (rim volume x 1.95) + (height variation contour x 30.12) - (corrected cup shape x 28.52) - 10.08, where corrected cup shape is cup shape + [0.0019 x (50 - age)]
LDF Bathija et al.12 : -3.72 - (5.57 x height variation contour) + (11.78 x RNFL thickness) - (4.37 x cup shape) + (1.85 x rim area)
• LDF Mardin et al.11 : -2.77 + (0.3 x rim area) + (3.70 x rim volume) + (4.30 x RNFL thickness) - (3.70 x cup shape)- (3.10 x cup volume) - (0.90 x cup area)
• LDF Iester et al.14 : (10.07 x cup area temporal inferior sector) - (7.02 x effective area temporal inferior sector) + (4.18 x mean height contour nasal sector) + (3.10 x mean height contour temporal sector) - (2.08 x peak height contour nasal superior sector) + (6.09 x cup shape) - (11.09 x rim volume temporal superior sector) - (8.05 x volume below surface temporal sector) + 1.83

We also developed and evaluated an LDF (called "current" LDF) that used all 83 parameters as input. This LDF was developed and tested, with 10-fold cross-validation used to reduce bias in developing and testing on the same samples (described later).

Neural Network Techniques
We evaluated the performance of three artificial neural network techniques for classifying eyes as glaucomatous or healthy. For all neural network techniques, all HRT parameters described earlier were included initially in the training set. Details and mathematical descriptions of the neural network techniques used have been described elsewhere by us and by others.19 20 21 22 23 24

Multilayer Perceptron.
The MLP, a feed-forward back-propagation network, is the most frequently used neural network technique in glaucoma research. Researchers have used this method to assess optic disc topography,8 15 to interpret and classify visual fields19 25 26 27 28 and to detect visual field progression.29 Briefly, MLPs are supervised learning classifiers that consist of an input layer (multiple HRT parameters, in our case), an output layer (glaucoma or not glaucoma, in our case), and one or more hidden layers that extract useful information during learning and assign modifiable weighting coefficients to components of the input layer. In the first (forward) pass, weights assigned to the input units and the nodes in the hidden layers and between the nodes in the hidden layer and the output, determine the output. The output is compared with the target output (binary glaucoma or no glaucoma). An error signal is then back propagated and the connection weights are adjusted correspondingly. During training, MLPs construct a multidimensional space, defined by activation of the hidden nodes, so that the two classes (glaucoma, not glaucoma) are as separable as possible. The separating surface adapts to the data.

We used a 10-unit MLP in the present study constructed in a commercial software program (Neural Network toolbox ver. 3.0 of Matlab; The MathWorks, Inc., Natick, MA). Input nodes fed into a 10-node hidden layer activated by hyperbolic tangent functions. Output was a single node with a logistic function for glaucoma (1) and healthy (0) eyes. Training was accomplished with the Levenberg-Marquant enhancement of back propagation. We evaluated MLPs with different numbers of units and found the 10-unit MLP performed best, as measured by performance of cross-validation.

Linear Kernel (SVM linear) and Gaussian Kernel Support Vector Machine (SVM Gaussian).
SVMs are newly developed techniques used for solving classification and regression problems. SVM architecture resembles the architecture of MLPs (input layer, hidden layer, output layer). During training, the SVM nonlinearly maps the training data to a high dimensional space where a hyper plane is fit that maximizes the margin of separation between classes while minimizing the generalization error (ability to generalize results from finite training set to data set), with the use of statistical learning theory. Constraints imposed on the construction of the separating surface result in a subset of training data that is involved in the decision function (called support vectors). The SVM attempts to split the positive and negative vectors to optimize the distance between the hyperplane and the nearest of the positive and negative examples. SVM linear and SVM Gaussian differ because they assume different distributions of input data. SVM linear uses linear mapping, resulting in a "dot product kernel" and SVM Gaussian uses unknown nonlinear mapping, resulting in a Gaussian kernel. Both SVM linear and SVM Gaussian have been used to classify eyes as glaucomatous or nonglaucomatous, based on visual field data.19

The SVM was programmed using the software program (Matlab, ver. 5.0; the MathWorks) and trained using Platt’s sequential minimal optimization algorithm. The programmer chose the parameters for penalty and the kernel by trial and error. The penalty used was C = 1.0.

Analysis
ROC curves for classifying eyes as glaucomatous or healthy were determined for all techniques. These curves describe the continuous relationship between sensitivity and specificity at specificities ranging from 0% to 100% and quantify the diagnostic accuracy of a test in a single number. An area under the ROC curve of 0.50 is equivalent to chance discrimination, and an area of 1.00 is equivalent to perfect discrimination. For SVMs and current LDF, 10-fold cross-validation was used to evaluate the classifiers. The glaucomatous and healthy eyes were each divided randomly into 10 approximately equal subsets. Ten mutually exclusive partitions were formed for cross validation (to measure the true rather than the estimated error rate) by combining one of the 10 healthy subsets with one of the 10 glaucoma subsets. One partition was used as the test set and the remaining nine partitions were combined to form the training set. The process was iterated, with each partition serving once as the test set. The results obtained for the 10 test sets were combined to generate a single ROC curve for each classification method. For MLP, cross-validation was similar, except eight partitions were used for training, one was used as a test set, and one was used as a stopping set to avoid overtraining. We provided sensitivities at specificities of 75% (representing moderate specificity) and 90% (representing high specificity), although this information is available in the graphic representations of ROC curves also presented. Finally, we reported the area under the ROC curve when specificity was 90% or more for the different techniques. These areas are bound by the ROC curve, the point at 100% specificity, and the line that passes through the point at 90% specificity and is perpendicular to the diagonal that represents chance discrimination. This information was provided to examine differences between techniques when specificity was high. The 90% specificity level was chosen because it theoretically forces the cases presumed to be the most difficult into the disease group by allowing only 10% of these cases into the healthy group.

We used the method of DeLong et al.30 to determine statistically significant differences in overall areas under the ROC curves.


    Results
 Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 References
 
Comparing Linear Discriminant Functions
Areas under the ROC curves (with sensitivities at 75% and 90% specificities) for all classification techniques evaluated are shown in Table 2 . The area under the ROC curve (± SE) for the best-performing LDF was for the current LDF (0.906 ± 0.02). The area for the best previously proposed LDF was for LDF Bathija et al.12 (0.890 ± 0.02), followed by LDF Mardin et al.11 (0.873 ± 0.02), LDF Iester et al.14 (0.860 ± 0.02), and the HRT classification by Mikelberg et al.10 (0.848 ± 0.02). Areas under the ROC curves of the current LDF, LDF Bathija et al., and LDF Mardin et al. were significantly greater than that of LDF Mikelberg et al. (all P = 0.02). No other statistically significant differences between areas under the ROC curves of proposed LDF were observed. ROC curves for the five LDFs are shown in Figure 1 . Areas under the curves when specificity was constrained from 90% to 100% were 0.184, 0.134, 0.143, 0.138, and 0.116, for current LDF, LDF Bathija et al., LDF Mardin et al., LDF Iester et al., and LDF Mikelberg et al., respectively.


View this table:
[in this window]
[in a new window]
 
Table 2. Area under the ROC Curve and Sensitivities

 


View larger version (40K):
[in this window]
[in a new window]
 
Figure 1. ROC curves (and areas under the curves) for the four previously proposed LDFs and current LDF. Areas for current LDF, LDF Bathija et al., and LDF Mardin et al. were significantly greater than the area for LDF Mikelberg et al.

 
Sensitivities at 75% specificity for current LDF, LDF Bathija et al., LDF Mardin et al., LDF Iester et al., and LDF Mikelberg et al., were 88%, 83%, 81%, 80%, and 81%, respectively. Sensitivities at 90% specificity were 81%, 67%, 70%, 69%, and 64%.

Comparing Neural Network Techniques
For MLP, the area under the ROC curve was 0.941 ± 0.01; for SVM linear, 0.938 ± 0.01; and for SVM Gaussian, 0.945 ± 0.01. No statistically significant differences between areas under the curves of neural network techniques were observed. ROC curves for the three neural network techniques are shown in Figure 2 . Areas under the curves when specificity was constrained from 90% to 100% were 0.178, 0.182, and 0.203, for MLP, SVM linear, and SVM Gaussian, respectively.



View larger version (30K):
[in this window]
[in a new window]
 
Figure 2. ROC curves (and areas) for the three neural network techniques investigated. No statistically significant differences between neural network areas under the ROC curve were observed.

 
Sensitivities at 75% specificity for MLP, SVM linear, and SVM Gaussian were 95%, 91%, and 92%, respectively; sensitivities at 90% specificity were 78%, 78%, and 83%, respectively.

Comparing Linear Discriminant Functions with Neural Network Techniques
Areas under the ROC curves were significantly higher for MLP, SVM linear, and SVM Gaussian than with all previously proposed LDFs (all P < 0.01), and with the current LDF (all P < 0.03). ROC curves for the best neural network (SVM Gaussian), the current LDF, and the best previously proposed LDF (LDF Bathija et al.) are shown in Figure 3 .



View larger version (33K):
[in this window]
[in a new window]
 
Figure 3. ROC curves (and areas under the curves) for the best LDF (Bathija et al., attribution in Figure 1 ), best neural network technique (SVM Gaussian), and the current LDF. The area for SVM Gaussian is significantly greater than that for both LDFs.

 
Optimizing Neural Network and LDF Results
The neural network technique that provided the largest area under the ROC curve when all HRT parameters were included as input to the training set was the Gaussian SVM. We performed feature selection both by sequential forward selection and sequential backward elimination of features31 to determine whether relying on more effective features and removing less effective features would improve the performance of a classifier as measured by area under the ROC curve. During forward selection, an optimum training (input) set was determined by starting with an empty subset and adding one input parameter at a time (e.g., the one that most increased the area under the curve in combination with the previously selected parameters) to the previously selected features until the area reached a maximum. During backward elimination, an optimal training set was found by starting with the full dimensional set from which the least effective input parameter was removed, one input parameter at a time (e.g., the one that resulted in the smallest increase in area under the ROC curve) until the maximum area was reached.

Figures 4 and 5 show that we achieved the optimal area under the ROC curve with either forward selection or backward elimination when we were using approximately 40% of the input parameters. These figures show areas under the ROC curve (y-axis) as a function of the number of HRT parameters in the training set (x-axis). The areas were maximized with a reduced dimension data set (subset of available input parameters) that contained an optimal combination of features determined by each optimization method, compared with using the full-dimensional feature set (all available input parameters). Using forward selection, the area under the ROC curve (± SE) increased from 0.945 (± 0.01) with all input parameters, to a maximum of 0.967 (± 0.01) with 31 input parameters. When the optimal feature set was analyzed at specificities constrained from 90% to 100%, the area under the ROC curve increased from 0.203 to 0.236. Sensitivity at 75% specificity increased from 92% to 97%, and sensitivity at 90% increased from 83% to 91%. When backward elimination was used, the area under the ROC curve increased to 0.965 ± 0.01 and reached its maximum with 32 input parameters. When specificity was constrained from 90% to 100%, the area was 0.213. Sensitivity at 75% specificity was 98% and sensitivity at 90% specificity was 85%. HRT parameters included in the optimized SVM Gaussian training set with both methods are shown in Table 3 .



View larger version (19K):
[in this window]
[in a new window]
 
Figure 4. Use of forward selection to determine an optimum training set for SVM Gaussian. The area under the ROC curve (y-axis) is shown as a function of the number of HRT parameters in the training set (x-axis). The training set is optimized at maximum area under the curve (0.976, n = 31 parameters).

 


View larger version (19K):
[in this window]
[in a new window]
 
Figure 5. Use of backward elimination to determine an optimum training set for SVM Gaussian. The area under the ROC curve (y-axis) is shown as a function of number of HRT parameters in the training set (x-axis). The training set is optimized at maximum area under the curve (0.965, n = 32 parameters).

 

View this table:
[in this window]
[in a new window]
 
Table 3. HRT Parameters Included in Optimized Training Sets for SVM Gaussian and Current LDF

 
In an attempt to maximize the performance of the current LDF, we optimized the training set by using the same methods described herein. When forward selection was used, the area under the ROC curve (±SE) increased from 0.906 ± 0.02, with all input parameters, to 0.960 ± 0.01, with 29 input parameters. The areas when specificity was constrained from 90% to 100% increased from 0.184, with all input parameters, to 0.213, with 29 input parameters. Sensitivity at 75% specificity increased from 88% to 95%, and sensitivity at 90% increased from 81% to 86%. When backward elimination was used, the area under the ROC curve increased to 0.961 ± 0.01, with 27 input parameters. When specificity was constrained from 90% to 100%, the area was 0.223. Sensitivity at 75% specificity was 95% and sensitivity at 90% specificity was 88%. HRT parameters included in the optimized current LDF training set with both methods are shown in Table 3 . The glaucomatous-healthy classification performance of the optimized current LDF was similar to that of the optimized SVM Gaussian, indicating that, with an optimal feature set, the data are linearly separable, and adaptive classifiers may not be necessary. Areas under the ROC curves for the optimized and full-dimensional current LDF and SVM Gaussian are shown in Figure 6 .



View larger version (36K):
[in this window]
[in a new window]
 
Figure 6. ROC curves (and areas) for optimized and nonoptimized SVM Gaussian and current LDF. The areas under the ROC curves for optimized SVM Gaussian and optimized current LDF were significantly greater than the areas for nonoptimized SVM Gaussian and nonoptimized current LDF.

 
Optimal Parameters within the Full-Dimensional Input
To determine some of the most informative HRT parameters, we identified a subset of input parameters from the full dimensional data set that most affected the area under the ROC by using forward selection and backward elimination.32 With each optimization method, input parameters were ranked from having the most (rank 1) to the least (rank 78) effect on the area under the curve when combined with other effective parameters. These ranks were plotted on a two-dimensional graph (forward selection rank on the y-axis, backward elimination rank on the x-axis). Those parameters closest to the origin were considered the most informative ones, because they presumably had the greatest influence on the area under the ROC curve with both optimization methods. This method was applied to both SVM Gaussian and current LDF (Figs. 7 8) . For SVM Gaussian, the three most informative parameters were peak height contour in the temporal inferior region, global cup shape, and disc area in the nasal region. For current LDF, the three most informative parameters were global cup shape, global rim volume, and cup area (area below reference) in the nasal superior region.



View larger version (48K):
[in this window]
[in a new window]
 
Figure 7. Identifying optimal parameters for full dimensional input for SVM Gaussian. Input parameters (see Table 1 ) are ranked from having the most (rank 1) to the least (rank 78) effect on area under the ROC curve with forward selection and backward elimination and are plotted on a two-dimensional graph. Parameters closest to the origin are considered the most informative ones because they presumably have the greatest influence on area under the ROC curve with both optimization methods.

 


View larger version (47K):
[in this window]
[in a new window]
 
Figure 8. Identifying optimal parameters for full dimensional input for current LDF with forward selection and backward elimination.

 

    Discussion
 Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 References
 
In our sample, all investigated HRT-based neural network techniques performed as well as or better than the HRT-based linear discriminant functions. ROC curves for nonoptimized neural network techniques ranged from 0.938 to 0.945, compared with 0.848 to 0.906 for LDF methods. Further, optimization of the feature set significantly increased discrimination ability, probably because of the removal of parameters that add information that has less value than the cost of including them in the training process. These results suggest that neural network classification techniques trained on HRT parameters are promising for discriminating between healthy eyes and those with mild to moderate glaucomatous visual field defects.

In the present study, the nonoptimized technique that resulted in the largest area under the ROC curve for discriminating between glaucomatous and healthy eyes (area under ROC curve = 0.945 for SVM Gaussian) yielded a sensitivity of 83% at 90% specificity. In previous work, Uchida et al.8 reported an area under the ROC curve of 0.94 and sensitivity and specificity of 92% and 91%, respectively, when using a back-propagation multilayer perceptron trained with nine global HRT parameters. These results are similar to the best optimized results from the present study (optimized SVM Gaussian: 91% sensitivity at 90% specificity). Severity of glaucoma in the patients in the present study was slightly higher than that of Uchida et al. (SAP mean deviation of -6.1 and -4.8 dB, respectively). Our study is the first to investigate the performance of SVMs trained on optical imaging data for discriminating between glaucomatous and healthy eyes.

Other studies have examined the success of individual HRT parameters, linear discriminant analyses, and specially developed parameters for classifying eyes as glaucomatous or healthy. For example, Iester et al.6 and Uchida et al.8 found the HRT cup shape measure to be the best individual parameter for identifying glaucoma. These authors reported areas under the ROC curve for glaucoma detection of 0.81 and 0.84, respectively, using this parameter. These areas are slightly higher than that reported by Zangwill et al.9 (0.78) for the same parameter.

The most frequently investigated HRT parameter-based LDF is that developed by Mikelberg et al.10 Using this model, reported sensitivity for detecting glaucomatous eyes (defined by abnormal visual fields and/or abnormal appearing optic discs) ranges from 42% to 92%, and reported specificity for detecting healthy eyes ranges from 84% to 96%.9 10 12 33 In the current study, sensitivity was 81% and 64% at 75% and 90% specificity, respectively. Using other HRT parameter-based LDFs for discriminating between glaucomatous and healthy eyes, Mardin et al.11 and Iester et al.,14 reported sensitivities of 84% and 70%, respectively, and specificities of 95% and 92%, respectively. In the present study, at a set specificity of 90%, we found a sensitivity of 70% with the Mardin et al. LDF and a sensitivity of 69%, with the Iester et al. LDFs. Finally, Bathija et al.12 reported a sensitivity and specificity of 62% and 94%, respectively, within a demographically similar sample with similar inclusion criteria and severity of glaucoma as in the current study.

Using non-standard HRT parameters to discriminate between glaucomatous and healthy eyes, Caprioli et al.34 reported a sensitivity of 83% and specificity of 85% (parapapillary slope derived from radial height measures around the disc), Iester et al.14 reported a sensitivity of 65% and specificity of 100% (measurement of retinal height differential), and Wollstein et al.35 reported a sensitivity of 84% and specificity of 96% (rim area adjusted for disc area). The maximum sensitivity and specificity reported in the present study were 91% and 90%, respectively (for optimized SVM Gaussian). Comparisons across studies are difficult, however, because of differences in population demographics, definition and severity of glaucoma, and differences in sensitivity and specificity at chosen cutoff values. We include this information to provide a context for our results.

In the current study, we identified HRT parameters that most affected the area under the ROC curve for discriminating between glaucomatous and healthy eyes with forward selection and backward elimination used in the SVM Gaussian and current LDF techniques. For both SVM and LDF techniques, global cup shape was an important parameter, identified by both forward selection and backward elimination. This finding is interesting, because other research has shown that cup shape is among the best individual parameters for discriminating between glaucomatous and healthy eyes.6 8 9 Other parameters identified by our techniques are novel. For instance, with the SVM Gaussian technique, peak height contour in the temporal inferior region and disc area in the nasal region were identified. It is possible that techniques without clinical bias may identify unexpected parameters that are important for discriminating glaucomatous from healthy eyes. The discordance in the most effective parameters with different classifiers may be explained by differences in the classifier reasoning process.

Two possible limitations of the current study were the lack of independent samples on which to develop and test current LDF and neural network techniques and the significant difference in age between healthy subjects and patients. Although we used cross-validation to train the neural network classifiers and the current LDF, part of the reason for the improved performance of these techniques compared with previously proposed LDFs might be that our techniques were trained and tested on groups with similar demographics and severity of glaucoma. We plan to test these techniques on outside populations. Because of the age difference between healthy subjects and patients, we did not include age as input when training the neural networks or developing the current LDF. The inclusion of age might allow the neural networks to classify eyes as glaucomatous or healthy based on age alone. This classification is clearly not practical from a diagnostic standpoint. When age was included in the training set, areas under the ROC curves for all neural network techniques and current LDF increased by 0.01 or less. These increases were not statistically significant. Sensitivities at the chosen specificity cutoffs increased by less than 5%. We also trained and tested the neural network techniques on a subset of our data in which age in both the healthy subjects and patients was constrained to between 40 years and 81 years. This subset was composed of 133 healthy subjects (mean age, 65.10 years) and 90 patients with glaucoma (mean age, 66.78 years). Age was not significantly different between groups (t-test, P > 0.10). Areas under the ROC curve for all neural network techniques and current LDF changed by 0.01 or less and no changes were statistically significant. Sensitivities at the chosen specificity cutoffs changed by less than 5%.

Although neural networks successfully discriminated between healthy and glaucomatous eyes in this study, these techniques incur one general criticism. Due to the complexity of the classifiers, they do not allow the interaction of important variables to be identified and measured. Other classification techniques, such as Bayesian networks, allow better assessment of the relative contribution of features.

In summary, neural network techniques were more successful at discriminating between glaucomatous and healthy eyes than previously proposed LDFs. This improvement suggests that neural network techniques show at least as much potential for use in diagnosis of glaucoma as linear discriminant techniques. In addition, support vector machines demonstrated better generalization performance, and therefore better classification performance than MLPs (see also Refs. 19 ,34 ,35 ). This result, coupled with the fact that SVMs are faster to train than MLPs, suggests that SVMs show superior potential for use in diagnosis of glaucoma when compared with MLPs.


    Footnotes
 
Supported by The Glaucoma Research Foundation (CB), and by National Eye Institute Grants EY13235 (MHG) and EY11008 (LMZ).

Submitted for publication February 4, 2002; revised June 5, 2002; accepted June 17, 2002.

Commercial relationships policy: N.

The publication costs of this article were defrayed in part by page charge payment. This article must therefore be marked "advertisement" in accordance with 18 U.S.C. §1734 solely to indicate this fact.

Corresponding author: Christopher Bowd, Hamilton Glaucoma Center, Department of Ophthalmology, University of California at San Diego, La Jolla, CA 92093-0946; cbowd{at}eyecenter.ucsd.edu.


    References
 Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 References
 

  1. Weinreb, RN, Lusky, M, Bartsch, DU, Morsman, D. (1993) Effect of repetitive imaging on topographic measurements of the optic nerve head Arch Ophthalmol 111,636-638[Abstract]
  2. Rohrschneider, K, Burk, ROW, Kruse, FE, Volcker, HE. (1994) Reproducibility of the optic nerve head topography with a new laser tomographic scanning device Ophthalmology 101,1044-1049[Medline][Order article via Infotrieve]
  3. Janknecht, P, Funk, J. (1994) Optic nerve head analyser and Heidelberg retina tomograph: accuracy and reproducibility of topographic measurements in a model eye and in volunteers Br J Ophthalmol 78,760-768[Abstract/Free Full Text]
  4. Chauhan, BC, LeBlanc, RP, McCormick, TA, Rogers, JB. (1994) Test-retest variability of topographic measurements with confocal scanning laser tomography in patients with glaucoma and control subjects Am J Ophthalmol 118,9-15[Medline][Order article via Infotrieve]
  5. Yucel, YH, Gupta, N, Kalichman, MW, et al (1998) Relationship of optic disc topography to optic nerve fiber number in glaucoma Arch Ophthalmol 116,493-497[Abstract/Free Full Text]
  6. Iester, M, Mikelberg, FS, Swindale, NV, Drance, SM. (1997) ROC analysis of Heidelberg Retina Tomograph optic disc shape measures in glaucoma Can J Ophthalmol 32,382-388[Medline][Order article via Infotrieve]
  7. Nakla, M, Nduaguba, C, Rozier, M, Joudeh, M, Hoffman, D, Caprioli, J. (1999) Comparison of imaging techniques to detect glaucomatous optic nerve damage [ARVO abstract] Invest Ophthalmol Vis Sci 40(4),S397Abstract nr 2089
  8. Uchida, H, Brigatti, L, Caprioli, J. (1996) Detection of structural damage from glaucoma with confocal laser image analysis Invest Ophthalmol Vis Sci 37,2393-2401[Abstract/Free Full Text]
  9. Zangwill, LM, Bowd, C, Berry, CC, et al (2001) Discriminating between normal and glaucomatous eyes using the Heidelberg retina tomograph, GDx nerve fiber analyzer, and optical coherence tomograph Arch Ophthalmol 119,985-993[Abstract/Free Full Text]
  10. Mikelberg, FS, Parfitt, CM, Swindale, NV, Graham, SL, Drance, SM, Gosine, R. (1995) Ability of the Heidelberg Retina Tomograph to detect early glaucomatous visual field loss J Glaucoma 4,242-247
  11. Mardin, CY, Horn, FK, Jonas, JB, Budde, WM. (1999) Preperimetric glaucoma diagnosis by confocal scanning laser tomography of the optic disc Br J Ophthalmol 83,299-304[Abstract/Free Full Text]
  12. Bathija, R, Zangwill, L, Berry, CC, Sample, PA, Weinreb, RN. (1998) Detection of early glaucomatous structural damage with confocal scanning laser tomography J Glaucoma 7,121-127[Medline][Order article via Infotrieve]
  13. Iester, M, Jonas, JB, Mardin, CY, Budde, WM. (2000) Discriminant analysis models for early detection of glaucomatous optic disc changes Br J Ophthalmol 84,464-468[Abstract/Free Full Text]
  14. Iester, M, Parfitt, CM, Swindale, NV, Mikelberg, FS. (1997) Sector-based analysis of Heidelberg Retina Tomograph (HRT) parameters in normal and glaucomatous eyes [ARVO abstract] Invest Ophthalmol Vis Sci 38(4),S835Abstract nr 3892
  15. Brigatti, L, Hoffman, D, Caprioli, J. (1996) Neural networks to identify glaucoma with structural and functional measurements Am J Ophthalmol 121,511-521[Medline][Order article via Infotrieve]
  16. Mikelberg, FS, Wijsman, K, Schulzer, M. (1993) Reproducibility of topographic parameters obtained with the Heidelberg Retina Tomograph J Glaucoma 2,101-103
  17. Zangwill, L, Bowd, C, Weinreb, RN. (2000) Evaluating the optic disc and retinal nerve fiber layer in glaucoma II: optical image analysis Semin Ophthalmol 15,206-220[Medline][Order article via Infotrieve]
  18. Bowd, C, Zangwill, LM, Blumenthal, EZ, et al (2002) Imaging of the optic disc and retinal nerve fiber layer: the effects of age, optic disc area, refractive error, and gender J Opt Soc Am A 19,197-207
  19. Goldbaum, MH, Sample, PA, Chan, K, et al (2002) Comparing machine learning classifiers for diagnosing glaucoma from standard automated perimetry Invest Ophthalmol Vis Sci 43,162-169[Abstract/Free Full Text]
  20. Rumelhart, DE, Hinton, G, Williams, R. (1986) Learning representations of back-propagation errors Nature 323,533-536
  21. Broomhead, DS, Lowe, D. (1988) Multivariable functional interpolation and adaptive networks Complex Syst 2,321-355
  22. Bishop, CM. (1995) Neural Networks for Pattern Recognition Clarendon Press Oxford, UK.
  23. Vapnik, V. (1998) Statistical Learning Theory Wiley New York.
  24. Vapnik, V. (2000) The Nature of Statistical Learning Theory 2nd ed. Springer New York.
  25. Spenceley, SE, Henson, DB, Bull, DR. (1994) Visual field analysis using artificial neural networks Ophthalmic Physiol Opt 14,239-248[Medline][Order article via Infotrieve]
  26. Lietman, T, Eng, J, Katz, J, Quigley, HA. (1999) Neural networks for visual field analysis: how do they compare with other algorithms? J Glaucoma 8,77-80[Medline][Order article via Infotrieve]
  27. Goldbaum, MH, Sample, PA, White, H, et al (1994) Interpretation of automated perimetry for glaucoma by neural network Invest Ophthalmol Vis Sci 35,3362-3373[Abstract/Free Full Text]
  28. Mutlukan, E, Keating, D. (1994) Visual field interpretation with a personal computer based neural network Eye 8,321-323
  29. Brigatti, L, Nouri-Mahdavi, K, Weitzman, M, Caprioli, J. (1997) Automatic detection of glaucomatous visual field progression with neural networks Arch Ophthalmol 115,725-758[Abstract]
  30. DeLong, ER, DeLong, DM, Clarke-Pearson, DL. (1988) Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach Biometrics 44,837-845[Medline][Order article via Infotrieve]
  31. Ripley, BD. (1996) Pattern Recognition and Neural Networks Cambridge University Press Cambridge, UK.
  32. Platt, JC. (1998) Fast training of support vector machines using sequential minimal optimization Scholkopf, B Smola, A eds. Advances in Kernel Methods: Support Vector Machines ,185-208 MIT Press Cambridge, MA.
  33. Broadway, DC, Drance, SM, Parfitt, CM, Mikelberg, FS. (1998) The ability of scanning laser ophthalmoscopy to identify various glaucomatous optic disk appearances Am J Ophthalmol 125,593-604[Medline][Order article via Infotrieve]
  34. Caprioli, J, Park, HJ, Ugurlu, S, Hoffman, D. (1998) Slope of the peripapillary nerve fiber layer surface in glaucoma Invest Ophthalmol Vis Sci 39,2321-2328[Abstract/Free Full Text]
  35. Wollstein, G, Garway-Heath, DF, Hitchings, RA. (1998) Identification of early glaucoma cases with the scanning laser ophthalmoscope Ophthalmology 105,1557-1563[Medline][Order article via Infotrieve]
  36. Platt, J, Cristianini, N, Shawe-Taylor, J. (2000) Large margin DAGs for multiclass classification Solla, SA Leen, TK Müller, K-R eds. Advances in Neural Information Processing 12,547-553 MIT Press Cambridge, MA.
  37. Roth, V, Steinhag, V. (2000) Nonlinear discriminant analysis using kernel functions Solla, SA Leen, TK Müller, K-R eds. Advances in Neural Information Processing 12,568-574 MIT Press Cambridge, MA.



This article has been cited by other articles:


Home page
Br. J. Ophthalmol.Home page
K A Townsend, G Wollstein, D Danks, K R Sung, H Ishikawa, L Kagemann, M L Gabriele, and J S Schuman
Heidelberg Retina Tomograph 3 machine learning classifiers for glaucoma detection
Br. J. Ophthalmol., June 1, 2008; 92(6): 814 - 818.
[Abstract] [Full Text] [PDF]


Home page
IOVSHome page
C. Bowd, J. Hao, I. M. Tavares, F. A. Medeiros, L. M. Zangwill, T.-W. Lee, P. A. Sample, R. N. Weinreb, and M. H. Goldbaum
Bayesian Machine Learning Classifiers for Combining Structural and Functional Measurements to Classify Healthy and Glaucomatous Eyes
Invest. Ophthalmol. Vis. Sci., March 1, 2008; 49(3): 945 - 953.
[Abstract] [Full Text] [PDF]


Home page
IOVSHome page
C. Boden, K. Chan, P. A. Sample, J. Hao, T.-W. Lee, L. M. Zangwill, R. N. Weinreb, and M. H. Goldbaum
Assessing Visual Field Clustering Schemes Using Machine Learning Classifiers in Standard Perimetry
Invest. Ophthalmol. Vis. Sci., December 1, 2007; 48(12): 5582 - 5590.
[Abstract] [Full Text] [PDF]


Home page
IOVSHome page
P. Naithani, R. Sihota, P. Sony, T. Dada, V. Gupta, D. Kondal, and R. M. Pandey
Evaluation of Optical Coherence Tomography and Heidelberg Retinal Tomography Parameters in Detecting Early and Moderate Glaucoma
Invest. Ophthalmol. Vis. Sci., July 1, 2007; 48(7): 3138 - 3145.
[Abstract] [Full Text] [PDF]


Home page
IOVSHome page
A. Coops, D. B. Henson, A. J. Kwartz, and P. H. Artes
Automated Analysis of Heidelberg Retina Tomograph Optic Disc Images by Glaucoma Probability Score
Invest. Ophthalmol. Vis. Sci., December 1, 2006; 47(12): 5348 - 5355.
[Abstract] [Full Text] [PDF]


Home page
IOVSHome page
M. Shunmugam and A. Azuara-Blanco
The quality of reporting of diagnostic accuracy studies in glaucoma using the heidelberg retina tomograph.
Invest. Ophthalmol. Vis. Sci., June 1, 2006; 47(6): 2317 - 2323.
[Abstract] [Full Text] [PDF]


Home page
IOVSHome page
M.-L. Huang and H.-Y. Chen
Development and Comparison of Automated Classifiers for Glaucoma Diagnosis Using Stratus Optical Coherence Tomography
Invest. Ophthalmol. Vis. Sci., November 1, 2005; 46(11): 4121 - 4129.
[Abstract] [Full Text] [PDF]


Home page
IOVSHome page
Z. Burgansky-Eliash, G. Wollstein, T. Chu, J. D. Ramsey, C. Glymour, R. J. Noecker, H. Ishikawa, and J. S. Schuman
Optical Coherence Tomography Machine Learning Classifiers for Glaucoma Detection: A Preliminary Study
Invest. Ophthalmol. Vis. Sci., November 1, 2005; 46(11): 4147 - 4152.
[Abstract] [Full Text] [PDF]


Home page
IOVSHome page
M. H. Goldbaum, P. A. Sample, Z. Zhang, K. Chan, J. Hao, T.-W. Lee, C. Boden, C. Bowd, R. Bourne, L. Zangwill, et al.
Using Unsupervised Learning with Independent Component Analysis to Identify Patterns of Glaucomatous Visual Field Defects
Invest. Ophthalmol. Vis. Sci., October 1, 2005; 46(10): 3676 - 3683.
[Abstract] [Full Text] [PDF]


Home page
IOVSHome page
C. Bowd, F. A. Medeiros, Z. Zhang, L. M. Zangwill, J. Hao, T.-W. Lee, T. J. Sejnowski, R. N. Weinreb, and M. H. Goldbaum
Relevance Vector Machine and Support Vector Machine Classifier Analysis of Scanning Laser Polarimetry Retinal Nerve Fiber Layer Measurements
Invest. Ophthalmol. Vis. Sci., April 1, 2005; 46(4): 1322 - 1329.
[Abstract] [Full Text] [PDF]


Home page
IOVSHome page
L. M. Zangwill, K. Chan, C. Bowd, J. Hao, T.-W. Lee, R. N. Weinreb, T. J. Sejnowski, and M. H. Goldbaum
Heidelberg Retina Tomograph Measurements of the Optic Disc and Parapapillary Retina for Detecting Glaucoma Analyzed by Machine Learning Classifiers
Invest. Ophthalmol. Vis. Sci., September 1, 2004; 45(9): 3144 - 3151.
[Abstract] [Full Text] [PDF]


Home page
IOVSHome page
C. Bowd, L. M. Zangwill, F. A. Medeiros, J. Hao, K. Chan, T.-W. Lee, T. J. Sejnowski, M. H. Goldbaum, P. A. Sample, J. G. Crowston, et al.
Confocal Scanning Laser Ophthalmoscopy Classifiers and Stereophotograph Evaluation for Prediction of Visual Field Abnormalities in Glaucoma-Suspect Eyes
Invest. Ophthalmol. Vis. Sci., July 1, 2004; 45(7): 2255 - 2262.
[Abstract] [Full Text] [PDF]


Home page
Arch OphthalmolHome page
F. A. Medeiros, L. M. Zangwill, C. Bowd, and R. N. Weinreb
Comparison of the GDx VCC Scanning Laser Polarimeter, HRT II Confocal Scanning Laser Ophthalmoscope, and Stratus OCT Optical Coherence Tomograph for the Detection of Glaucoma
Arch Ophthalmol, June 1, 2004; 122(6): 827 - 837.
[Abstract] [Full Text] [PDF]


This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when eLetters are posted
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow