|
|
||||||||
1From the Department of Ophthalmology, and the 2Vision Cooperative Research Center, University of Melbourne, East Melbourne, Australia; the 3Department of Psychiatry, The Australian Centre for Posttraumatic Mental Health, University of Melbourne, Heidelberg, Australia; and 4Centre for Health Economics, Monash University, Clayton, Victoria, Australia.
| Abstract |
|---|
|
|
|---|
METHODS. Focus groups of the visually impaired elicited key concepts. Based on these and previous research, 33 items were generated. These were administered to visually impaired adults (n = 70) and a representative sample of unimpaired adults (n = 86). The item bank was reduced through examination of item properties, exploratory factor (EFA), item response theory (IRT), and structural equation modeling (SEM) analyses. The resultant model was confirmed through administration to a second sample of participants.
RESULTS. Focus group themes included physical well-being, social well-being, independence, self-actualization, emotional well-being, and planning and organization. Poorly performing items were eliminated on basic psychometric properties, including failure to discriminate. Next, EFA loadings were used to select items. Twelve items survived. To minimize redundancy, IRT analysis and SEM reduced the VisQoL item pool to six items (Cronbach
= 0.88). To confirm this model, these items were then administered to an additional 218 participants; 35% with a vision impairment. A pooled SEM analysis showed the model to have very good fit properties (root mean square error of approximation [RMSEA] = 0.000). A preliminary test of the model against visual acuity showed a significant monotonic relationship.
CONCLUSIONS. The short 6-item VisQoL has excellent psychometric properties as a simple summative instrument. It can be used in its present state as a condition-specific outcome measure for the evaluation of healthcare interventions for the visually impaired. The descriptive model is also suitable for generating utility values for the economic evaluation of vision-related programs and services.
The global goal of eye-care and vision-related rehabilitation programs and services is to improve the QoL of visually impaired people. Therefore, to assess accurately the value of a service or program, the impact on QoL should be considered, as well as the consumption of limited healthcare resources. Costutility analysis enables this assessment because it expresses the value of outcomes (typically improvements) in a universal unit, the quality of life-adjusted year (QALY) and the cost of providing the services. It is a critical tool in evidence-based medicine, as it considers both the quality and the efficiency of healthcare delivery, expressed as the cost per QALY gained from a service or program.
The long tradition in ophthalmology of using objective psychophysical measures has served the field well, but is no longer considered sufficient. For several years, there has been an increasing demand for the inclusion of subjective instruments to indicate gains in health-related quality of life (HRQoL) in such a way that these gains can be compared with gains in other health fields. When gains are comparable, rational decisions about the allocation of scarce healthcare resources can be made.
HRQoL measures are designed to evaluate the impact that health has on QoL and the positive or negative impact of medical care. These are useful measures because they are not restricted to a particular area of health that facilitates comparisons across different conditions. A widely used generic HRQoL measure is the EuroQoL Health Questionnaire (EQ5D).9
However, generic HRQoL measures may lack the sensitivity of disease- or condition-specific measures.10 11 12 For instance, generic measures may be less responsive to clinically relevant changes in vision.13 In addition, there are possibly some QoL issues that are specific to vision loss and hence are not adequately represented in a generic measure.12 Several vision-related QoL measures have been developed such as the Visual Function (VF)-14 for measuring visual functioning in cataract patients.14 Other vision-specific instruments include the VCM1 (vision-related quality of life questionnaire),15 the National Eye Institute Visual Function Quotient (NEI-VFQ),16 the Activities of Daily Vision Scale (ADVS),17 and the Impact of Vision Impairment profile (IVI).2 For the reasons just outlined it is widely accepted that studies should include both a generic and vision-specific measure. One allows comparison across conditions, and the other provides greater sensitivity to changes in vision-related QoL.
However, for economic evaluation, particularly costutility analysis, these measures are limited in their suitability, as they do not generate utility values. A utility value is an indirect measure of a persons QoL based on the persons preference for a particular health state. In other words, it quantifies the value of QoL associated with a particular health state. Generally, a weighted index of preference is determined on a scale from 0.00 (death) to 1.00 (perfect or very good health). When this value is known over time, it can be used to compute QALYs (i.e., QALYs = utility value x length of time gained). QALYs can be used in costutility analyses.
There are two ways that utilities can be obtained directly from respondents: by using the time-tradeoff (TTO) or by standard gamble (SG). TTO requires people to imagine living a fixed number of years with a particular health condition, and then to indicate how many of those years of life they would be willing to trade to have perfect health. SG determines the level of risk people are willing to take for a treatment that would give them perfect health. Both these direct methods have been used to determine vision-related utility values. For instance, one study reported a mean utility value of 0.72 for people with age-related macular degeneration with a TTO and 0.81 using the SG.18 Utility values were also obtained for people with diabetic retinopathy,19 as well as those who were blind.20 However, these direct utility values were based on a single utility model. For instance, the patients with diabetic retinopathy were asked how many of their remaining years they would be willing to trade to have perfect vision for the rest of their lives.19
Given that the impact of vision on QoL is multifaceted, to get a more accurate reflection of the overall value of vision, a multiattribute utility (MAU) model would be preferable. This is the second way utilities can be obtained. In an MAU instrument, the different parts of life that may be affected by a health condition are separately assessed, and then they are combined into an index that is presented on a lifedeath scale where 0.00 represents a death-equivalent state and 1.00 a perfect HRQoL state. There are several generic health utility measures, such as the Assessment of Quality of Life instrument (AQOL),21 the EQ5D,9 and the Health Utilities Index III.22 However, these generic utility measures may not be sufficiently sensitive to vision-related QoL; hence, their use in the evaluation of eye care and vision-related rehabilitation programs and services may be limited. A vision-specific utility measure would overcome this difficulty.
Due to these limitations in existing MAU measures, there is currently no instrument suitable for generating utility values for the purposes of costutility analyses of vision-related programs and services. In this project, we sought to address this deficit by the development of a multiattribute vision-related utility measure, referred to as the Vision and Quality of Life Index (VisQoL).
| Methods |
|---|
|
|
|---|
The key questions asked during the focus groups probed the effect of vision on self-care, mobility in and out of the home, participation in work and leisure activities, interaction with other people, and sense of oneself. There was also a question on which area of life vision has had the greatest impact. The concept of TTO was also introduced in the discussions. Grounded theory techniques were used to analyze transcripts and observer notes from the discussions. They were examined to identify distinct statements made, which were then coded into appropriate categories.
Using the results from the focus groups and from previous research, the second step in the development of the VisQoL was to create an item bank. The desired number of items for the final VisQoL scale was 6 to 10. A database of 33 items was developeda ratio of approximately 4:1 with the desired scale size.
The third step was to determine the suitability and validity of the items in the item bank for inclusion in VisQoL and to reduce the number of items from 33 to approximately 6 to 10. Participants in this second stage were chosen to represent two groups: (1) people who were visually impaired, defined as visual acuity (VA) <20/30 in the better eye, and (2) those deemed nonvisually impaired, defined as VA 20/20 or better in both eyes. All participants were 18 years of age or over.
People with impaired vision were recruited from the Royal Victorian Eye and Ear Hospital (RVEEH). Eligible participants were given the option of completing the questionnaire in the waiting room or taking it home and returning it via the reply-paid envelopes provided. In addition, participants were able to seek assistance from the research team, a family member, or a friend, if necessary to complete the questionnaire. The availability of assistance was important, as some participants were unable to read the questionnaire due to vision impairment. People examined and found to have normal vision from a population-based study, the Vision Impairment Project (VIP)1 were sent an invitation letter, explanatory statement, the VisQoL item bank, and a consent form by mail. Participants who chose to participate were asked to return the completed questionnaire and signed consent form using the reply-paid envelope.
To reduce the item bank to the final structure, basic psychometric properties were examined. In addition, exploratory factor (EFA), reliability, item response theory (IRT), and structural equation modeling (SEM) analyses were conducted.
The fourth and final step in the development of VisQoL was to confirm the structural equation model obtained in the pilot study with a second sample group. This second group was recruited using the same methods as in the pilot studythat is, people with impaired vision were recruited from the RVEEH, and those with normal vision were recruited from the VIP study. SEM was used with this sample to confirm the construction sample model. Because there are known difficulties with SEM ADF (asymptotic distribution-free) models constructed in AMOS,23 which may have provided misleading fit estimates, we pooled the two samples for a final confirmatory analysis.
Ethics approval for the study was granted by the Human Research and Ethics committee of the RVEEH. Informed, written consent was obtained from each participant. The research study adhered to the tenets of the Declaration of Helsinki.
General data analyses were performed on computer (SPSS ver. 11.5; SPSS Chicago, IL),24 the structural equation modeling was performed with AMOS (ver. 4.0),25 and the item-response theory analyses with Conquest.26
| Results |
|---|
|
|
|---|
Common issues were identified from the focus groups that were specific to vision impairment. Across the three focus groups, both intrinsic and mediating factors were revealed to facilitate participation and perceived QoL (Table 1) . Intrinsic factors were defined as issues concerning the self, such as independence, social well-being, emotional well-being, physical well-being, and self-actualization. Mediating factors, such as planning, organization, and strategic development, were defined as facilitating QoL through participation but were not an internal process or state of well-being.
|
|
20/20 in both eyes (Table 3) . Of the 70 people with a vision impairment, 18% had glaucoma as the main eye condition, 48% had a retinal condition, and 34% a corneal condition.
|
To reduce the item pool to approximately two representative items from each of the dimensions, iterative unconstrained factor analyses were conducted. In each case, the procedure was to delete the item with (1) the poorest psychometric properties, (2) the lowest factor loadings, and (3) the lowest pooled principal component analysis loading. At the end of the iterations, 13 items were retained in the item pool (Table 4) .
|
) were conducted on the remaining 13 items. Results revealed that all items loaded on the one factor (Table 5) . The high Cronbach
of 0.96 suggested redundancy among the items.
|
IRT examines the item degree of difficulty among a homogenous set of items for which the assessment scale reflects respondents underlying traits. Considering the nature of the data in this studythat is, the ordinal Guttman response scales rather than dichotomous responseswe perforemd the IRT analysis by using a polytomous partial credit model.27
Figure 1 displays all 27 items that were not rejected on the basis of their psychometric properties (listed in Table 4 ). The items marked in bold are those 13 items that were retained after the EFA procedure listed in Table 5 . The unobserved construct (a respondents ability to endorse responses, derived from their vision-related QoL impairment experiences) and the item difficulty are mapped on Figure 1 . Positive logit values suggest that these are items on which it is easy for respondents to endorse losses, whereas negative logits suggest respondents find it difficult to endorse losses on those items.
|
To remove this redundancy, the 13 items listed in Table 4 were examined using ADF SEM, as item responses were nonnormally distributed. The evidence from the EFA indicated that the model was unidimensional, therefore an iterative procedure was followed. The start model included all 13 items, with items progressively removed. The four criteria for improved fit were a significant reduction in the
2 value, improvement in the comparative fit index (CFI; the accepted value was >0.90), and an improvement in the root mean square error of approximation (RMSEA; the accepted value was <0.05).
After iteration, the final model consisted of six items representing the six dimensions (refer to the Appendix for final items). It consists of items 5, 7, 11, 25, 26, and 28. The SEM statistics for the model indicated excellent fit properties:
2 = 6.359, df = 8; CFI = 1.000, RMSEA = 0.000. The Cronbach
was 0.88.
The VisQoL descriptive system items were examined using partial-credit IRT. The results are given in Table 6 . This shows a wide range of weighted T-fit values, perhaps reflecting the diverse nature of the items within the VisQoL (not surprising, since these were originally chosen to represent different aspects of life). The worst-fitting item was that measuring organizing activities. That all item-difficulty threshold estimates are positive reflects that, in the community sample, few persons endorsed item response levels other than the top level.
|
|
2 = 11.91, df = 9; CFI = 0.972 (>0.90), RMSEA = 0.039 (<0.05).
As noted in the Methods section, however, there are known difficulties with AMOS ADF models in which the sample sizes are under
200 cases.23 It follows that the two separate SEM models reported earlier (for the pilot and confirmatory samples) may be unstable. For verification, therefore, SEM was repeated with the pooled sample. The results are given in Figure 2 , which shows excellent fit of the model to the data.
|
The obtained score range was 0 to 27. The mean for the community sample was 2.99 ± 2.44 (SD) compared with 9.99 ± 6.15 for the vision-impaired sample (ANOVAtransformed data, F = 236.35, df = 1372, P < 0.01). When examined by visual acuity, the results shown in Figure 3 were obtained, showing a monotonic relationship between visual acuity and VisQoL scores.
|
| Discussion |
|---|
|
|
|---|
The items were drawn from the broad areas of social, emotional, and physical well-being; independence; self-actualization; and planning and organization. These categories were identified in the focus groups and were consistent with those of other QoL measures.9 21 They also fit within the World Health Organization (WHO) description of good health as being due to the effect of physical, emotional, and social factors.28 An additional category was identified as being part of the intrinsic nature of QoL, self-actualization.
Two important features of this research were the use of patients views and the application of a mixture of classic and modern test theory. Regarding the use of focus groups to determine the underlying constructs to be measured, the steps we followed were consistent with those outlined by the WHO Quality of Life Group (WHOQOL Group) for QoL instrument construction.28 29
Other than for the AQoL,30 there is little evidence that MAU instruments have incorporated the patients perspective. Generally, they have been designed around the concerns of health clinicians, researchers, and policy makers. Fully conscious of the work of the WHOQOL Group, we set out to incorporate views of people with vision impairment. None of the psychometric tests used in the construction of the VisQoL suggested any difficulties with this procedure. Because it is patients whose lives are affected by healthcare interventions and resource allocation decisions, the procedures we followed to include their perspective suggests strong ecological validity for the VisQoL.
A second feature of this research project was the use of both classic and modern test theory practices to select items for inclusion in the VisQoL. As the results showed, classic approaches to item reduction enabled the removal of most items, but failed to discriminate adequately between items with similar properties. Essentially, the classic test construction methods resulted in a bank of items with equivalent characteristics, whereas the available tests were insufficient to eliminate double-counting. We suspect that this situation may be common, because it arises because of the very properties of the tests themselves which rely on correlations between items. The use of IRT and SEM enabled us to examine these issues in more detail and to select a very parsimonious group of representative items for inclusion. This ability of modern test construction procedures is one that has wide applicability elsewhere. That almost identical results were obtained with a second construction sample suggests the robustness of the procedures used in the construction of the VisQoL. We should point out, however, that classic test construction axioms regarding sampling and modern test construction techniques are not fully compatible, as shown in the IRT modeling of the descriptive system probability thresholds (see Table 6 and Fig. 1 ). That the thresholds are all positive logits reflects the effect of including both the community and visually impaired samples. Although a mixture of classic and modern techniques have been used elsewhere,31 32 33 as far as we are aware, this is the first time that this approach has been applied in the MAU instrument field. We believe that it could be useful in future instruments.
The final six items all load on the one vision-related QoL factor, whereas the results of the SEM indicate that they are measuring different aspects of vision-related QoL. Hence, it is likely these items form a unidimensional parsimonious measure but without extensive double-counting within the descriptive system. This is important, as it allows for the further development of a multiattribute approach to the measurement of vision-related QOL without violating utility theory axioms.34 Therefore, these six items represent a measure of vision-related QoL that is highly suitable for further development into a vision-related utility instrument.
Even so, as this article shows, the VisQoL has excellent psychometric properties as a simple summative instrument which can be used in its present state as a condition-specific outcome measure for the evaluation of healthcare interventions for the visually impaired.
| Appendix 1 |
|---|
|
|
|---|
|
| Footnotes |
|---|
Submitted for publication November 29, 2004; revised May 13, 2005; accepted September 14, 2005.
Disclosure: R. Misajon, None; G. Hawthorne, None; J. Richardson, None; J. Barton, None; S. Peacock, None; A. Iezzi, None; J. Keeffe, None
The publication costs of this article were defrayed in part by page charge payment. This article must therefore be marked "advertisement" in accordance with 18 U.S.C.
1734 solely to indicate this fact.
Corresponding author: Jill Keeffe, Center for Eye Research Australia, University of Melbourne, 32 Gisborne St., East Melbourne 3002, Australia; jillek{at}unimelb.edu.au.
| References |
|---|
|
|
|---|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |