- Citado por SciELO
- Citado por Google
- Similares en SciELO
- Similares en Google
versión impresa ISSN 0213-6163
Eur. J. Psychiat. vol.25 no.2 Zaragoza abr.-jun. 2011
Robert E. Mann*,**; Joyce T.W. Cheung***; Anca Ialomiteanu*; Gina Stoduto*; Vincy Chan****; Christine M. Wickens*; Kari Ala-leppilampi*,**; David Goldbloom*****,******; Jürgen Rehm*,**,******
* Social and Epidemiological Research, Centre for Addiction and Mental Health, Toronto, Ontario. Canada
** Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario. Canada
*** Health Studies, University of Toronto, Toronto, Ontario. Canada
**** Department of Psychology, University of Toronto, Toronto, Ontario. Canada
***** Education and Public Affairs, Centre for Addiction and Mental Health, Toronto, Ontario. Canada
****** Department of Psychiatry, Faculty of Medicine, University of Toronto, Toronto, Ontario. Canada
a Funding and support: This research has been supported by funding from the Ontario Ministry of Health and Long-Term Care. The views expressed here do not necessarily reflect those of the Ministry.
Background and Objectives: Our study explored the validity of different threshold values on the 12-item version of the General Health Questionnaire (GHQ12) for estimating the prevalence of anxiety and mood disorders (AMD) in Ontario population survey data.
Methods: Data were drawn from the 2003, 2004 and 2006 cycles of the CAMH Monitor (N = 7,126), an ongoing general population survey of Ontario adults aged 18 and older, which includes the GHQ12. The concordance of different threshold values on the GHQ12 for determination of AMD with a criterion based on individuals who were prescribed either anti-anxiety or anti-depressant drugs in the past 12 months and who reported 14 or more mentally unhealthy days in the past 30 days was examined using receiver operator characteristic (ROC) analysis.
Results: Concordance between the GHQ12 determination of AMD and the criterion measure reached "moderate" levels. ROC analysis revealed an area under the curve (AUC) of 0.89. At a GHQ12 threshold value of 4, the specificity and sensitivity values obtained were 0.92 and 0.71, respectively. Also at that value, the estimated prevalence of AMD was nearly identical to that seen in recent Canadian studies using the CIDI.
Conclusions: These analyses suggest that the GHQ12 may be suitable for providing a proxy measure of AMD for epidemiological and surveillance purposes. A threshold score of 4 seems to be most suitable for these purposes when using Canadian data.
Key words: GHQ12; Anxiety disorder; Mood disorder; Survey; Validity; Prevalence.
Anxiety and mood disorders (AMD) are common, treatable psychiatric conditions that are linked to other health conditions and create a tremendous burden for individuals and society1-3. However, epidemiological studies, including surveillance studies, of AMD present significant challenges. Epidemiologic research that has attempted to identify population prevalence of these disorders has employed comprehensive diagnostic instruments, such as the World Health Organization's Composite International Diagnostic Interview (CIDI)4, that are valuable but nevertheless very costly and time consuming to administer in a survey format5. Several authors5,6 have pointed to the value of briefer, surrogate measures of psychiatric problems that can provide important population-level information for ongoing surveillance and other epidemiologic purposes.
The General Health Questionnaire (GHQ) was developed as a screening instrument for detecting psychological disorder in clinical and other settings7,8. It has been widely used for this purpose, and has proven generally useful for the detection of psychiatric distress in a variety of circumstances, including in general population surveys9,10. One challenge in using the GHQ is linking it to specific psychiatric conditions. Goldberg et al.8 examined the validity of the full GHQ and the 12-item version (GHQ12) by assessing their ability to detect anxiety, mood and somatisation disorders as defined based on administration of the CIDI, with diagnoses based on the International Classification of Diseases (ICD-10) (the data were also validity checked with DSM-IV and were nearly identical). The ability of both questionnaires to detect these disorders in a multi-site international sample was described as "uniformly good". However, the GHQ12 does not contain any of the items from the full GHQ assessing physical symptoms, and instead all the GHQ12 items deal with depression and anxiety issues. Thus, it is more appropriate to view the GHQ12 as assessing anxiety- and mood-related symptomatology, and not somatoform disorders.
Previous validity studies have compared GHQ12 scores to various gold standards for defining caseness (based on a diagnostic interview such as the CIDI or clinician diagnosis). Goldberg and colleagues8 reviewed previous studies on the validity of the GHQ12 to define caseness in primary care samples, and reported threshold scores on the instrument ranging from 1 to 8 (mode = 3) across studies. Median sensitivity and specificity values (range in brackets) of 83.7% (67.5-93.5) and 79.0% (59.0-93.0) were reported. The results of this and subsequent studies with clinical samples have continued to provide good evidence for the validity of the GHQ12 in detecting AMD, although several threshold values for AMD have been suggested8,11-13.
Recent studies from several countries assessed the usefulness of the GHQ12 to detect AMD in general population samples. Doi and colleagues14 validated a Japanese version of the instrument in a representative sample of 1808 adult Japanese, and found that the instrument showed good internal reliability in that sample, with Cronbach's alpha coefficients of 0.83 for men and 0.85 for women. In the Northern Rivers Mental Health Study of a large Australian general population cohort, among those not diagnosed with AMD at baseline, elevated GHQ12 scores predicted CIDI-based AMD diagnoses two years later15. Furukawa et al.16 examined the ability of the GHQ12 to detect AMD based on the CIDI in the Australian National Survey of Mental Health and Well-Being. They observed that the performance of the instrument was good, with an AUC of 0.80, although performance of two other screening instruments, the K6 and K10, appeared so-mewhat better with AUCs of 0.89 and 0.90 respectively. Using data from the same survey, Donath17 examined the ability of the GHQ12 to detect CIDI-defined AMD and neurasthenia. For the standard scoring method, the optimal threshold in terms of trade-off between sensitivity and specificity was 0/1, and the AUC was 0.79. Bijl et al.18, in a population survey in the Netherlands, reported very close correspondence between prevalence estimates for AMD based on the GHQ12 and on the CIDI.
While reports on the validity of the GHQ12 for identifying AMD in population surveys from several countries have appeared, no Canadian studies have been reported. The purpose of this study is to explore the ability of the GHQ12 to provide valid estimates of the prevalence of AMD for epidemiologic and surveillance purposes in Canadian data. We assess the ability of the GHQ12 in a representative sample of the Ontario adult population to identify respondents treated for AMD and to explore what threshold value would be most appropriate for estimating the prevalence of AMD in these data. Our interest is to explore the use of the GHQ12 as a proxy measure of treated AMD for epidemiological purposes.
Data are based on the CAMH Monitor, an ongoing cross-sectional telephone survey of Ontario adults (18 years or older) conducted by the Centre for Addiction and Mental Health and administered by the Institute for Social Research at York University. The survey uses random-digit-dialing methods via Computer Assisted Telephone Interview (interview length average 25 minutes). The CAMH Monitor each year consists of 12 independent monthly surveys with 200 completions expected each month. The design employs a two-stage probability selection procedure. Each month a sampling frame of all active area codes and exchanges in Ontario is provided by the ATT Long Lines Tape. Within each regional stratum, a random sample of telephone numbers is selected with equal probability in the first stage of selection (i.e., households). Within selected households, one respondent aged 18 or older who can complete the interview in English or French is selected according to the most recent birthday of household members. Across years, response rates ranged from 53% to 61%; which is considered good for surveys of this nature19. For current study purposes, 2003, 2004 and 2006 data were merged into one dataset (N = 7,126). For additional sampling design details see Ialomiteanu and Adlaf20.
The 12-item version of the General Health Questionnaire (GHQ12) in the CAMH Monitor (see Appendix) was scored in the standard manner8; Likert responses (1, 2, 3, 4) were recoded to "0, 0, 1, 1", where 1 represents a response indicating "somewhat more than usual" or "a lot more than usual" experience of a symptom. The GHQ12 items correspond well with many of the CIDI items for mood and anxiety disorder. The internal consistency of the GHQ12 in our study sample was high (Cronbach's alpha coefficient 0.833).
Respondents were asked about their use of anti-depressant and anti-anxiety drugs: "In the past 12 months, have you taken any prescription medication to treat depression?" "In the past 12 months, have you taken any prescription medication to reduce anxiety or panic attacks?" Respondents were also asked about the number of days they experienced mental health problems in the past 30 days: "Now thinking of your mental health, which includes stress, depression, and problems with emotions, for how many days during the past 30 days was your mental health not good?"
The results in this paper are based on "valid" responses (n's) such that missing data (i.e. "don't know" responses and refusals) were excluded from analyses. SPSS 15.0 software was employed for all analyses. The percentages reported are based on the weighted sample size and are considered representative for the population surveyed. We examine the proportion of the sample who would be identified by the GHQ12 as having an anxiety and mood disorder (AMD) at threshold scores ranging from 1 to 12 with breakdowns by age group and gender using chi-square analysis.
The criterion "case" group for validity comparison purposes consisted of respondents who reported treatment for AMD, as defined by reporting being prescribed either anti-depressant or anti-anxiety drugs in the past 12 months, and experiencing 14 days or more of mental health problems in the past 30 days. Constructing the criterion group in this manner provides substantial confidence that those included would have been diagnosed with AMD by a clinician in the past year. However, it also means that the criterion group will likely underestimate the true 12-month prevalence of AMD in the sample, in view of the known high rates of undetected disorder21. Receiver operator characteristic (ROC) analysis was used to examine the concordance of GHQ12 thresholds ranging from 1 to 12 for determination of treated AMD cases.
Table 1 presents the characteristics of the sample. A total of 2.1% of the sample were included in the case group of treated AMD respondents (i.e., those who reported 14 or more days of mental distress in the preceding 30 days, and being prescribed either anti-anxiety or anti-depressant medication in the past 12 months).
Table 2 presents the proportion of the sample who would be identified by the GHQ12 as having an anxiety and mood disorder (AMD) at threshold scores of 1 to 12, and this proportion ranges from 33.3% at a threshold of 1 to 0.3% at a threshold of 12. The table also presents breakdowns by age group and gender. The proportion of females identified with AMD is higher than the proportion of males at all thresholds. Similarly, the proportion identified with AMD is highest in the youngest age category and declines with age, for all threshold values.
Table 3 presents the sensitivity and specificity values for thresholds ranging from 1 to 12. Sensitivity values decrease from 0.9 at 1 to 0.09 at 12. Specificity values range from a high of 0.99 for thresholds ranging from 9 to 12, to a low of 0.68 at a threshold of 1. Figure 1 presents the ROC curve plotting sensitivity versus 1- specificity. The Area Under the Curve (AUC) value is 0.89. According to Akobeng22, AUC values of 0.7 to 0.9 reflect a test with moderate accuracy, and values of 0.9 and above reflect high accuracy. Specificity results are generally strong, indicating that the instrument performs well in identifying non-cases. Results for sensitivity are more modest, declining as threshold increases and ranging from 0.90 to 0.09. In general, these results suggest that the GHQ12 shows good validity as a detector of treated AMD cases in this population, and thus has substantial promise as a proxy measure of AMD prevalence in survey data.
One method to suggest an optimal cutoff level is to identify the cutoff that produces the maximum of the product of the sensitivity and specificity22, and this product reaches a maximum value at a threshold of 4. Prevalence matching involves comparing prevalence estimates obtained at each threshold score with known prevalence rates. At a cutoff of 4, the proportion that would be identified with treated AMD (9.4%) exceeds the proportion of the sample in the criterion group (2.1%). Interestingly, Rush et al.23 report that the 12-month population prevalence of any mood or anxiety disorder in the Canadian population is 8.4%, based on the version of the CIDI included in the Canadian Community Health Survey 1.2. The corresponding 12-month prevalence estimates for males and females were 6.4% and 10.3%, respectively. These prevalence estimates are very close to the prevalence estimates that would result from setting the GHQ12 threshold for defining caseness at 4, which would provide prevalence estimates of 9.4% for the full sample, and 7.7% and 10.6% among males and females, respectively. These observations suggest that a GHQ12 threshold value of 4 may be providing a reasonable estimate of the prevalence of both treated and untreated AMD in a Canadian adult population.
This study assessed the ability of the GHQ12 to identify adults treated for Anxiety and Mood Disorder (AMD) and explored what threshold value would be most appropriate for this use, based on an Ontario population survey sample. While these data are of interest, several limitations must be kept in mind. First, the data are based on self-report and thus may be affected by recall bias. However, reviews of self-report methods suggest that although surveys may underestimate true rates of problems (e.g., drug and alcohol abuse), they are still regarded as the best means to estimate such behaviours for public health purposes24.25. Also, although the response rate for the survey is considered good for surveys of this nature19, we cannot be certain that non-respondents would have responded the same way as respondents in this study. A related issue here is that the survey relies on sampling households with traditional land-line telephones. Statistics Canada26 estimated that in 2007, 6.4% of households reported having cell phones only. Not including cell phone only households may influence the representativeness of the sample. In particular, younger individuals may be more likely to have cell phones only and thus may be under-represented in the sample reported here. It is also the case that our criterion group likely underestimates the true prevalence of AMD in the sample. Our criterion group was constructed in part using medication data reflecting clinical decisions by practitioners. Underestimation of the "true" prevalence of AMD based on clinician diagnoses has been noted3, and other studies have noted "poor" agreement between clinician diagnoses of AMD and those based on the CIDI27. Finally, we note that because this work involved an existing database our selection of measures was restricted to those available. As a result, because of variation in time periods reflected in our measures of AMD (anxiety or depression prescription medication use in the past 12 months, poor mental health in the past 30 days, past few weeks for the GHQ12), we cannot be certain how closely the "cases" defined by the GHQ12 in our study would have corresponded to "cases" identified by the CIDI or similar measures. This is an important concern in the literature on measures to assess AMD in the population more generally27.
Keeping these limitations in mind, the results are of substantial interest. Population-based estimates of the prevalence and risk factors for major psychiatric disorders provide very important information for understanding, preventing and treating these disorders. Ongoing monitoring of population rates of AMD would be very valuable for surveillance and other epidemiological purposes, but this information is very difficult to obtain with current methods. Ideally, prevalence estimates are based on validated and comprehensive diagnostic processes. However, these processes are costly and time-consuming. Instruments developed for these purposes, such as the CIDI, require substantial time or special instructions for administration that may not be feasible in many epidemiological surveys. Thus, there would be value in a brief, easily administered instrument that can provide a population estimate of AMD for surveillance purposes and epidemiological research on prevalence and risk factors. The analyses reported here were undertaken to determine if the GHQ12 could be used for this purpose.
The results suggest that it is possible to use the GHQ12 to provide estimates of the prevalence of treated AMD, and perhaps untreated AMD, in the Ontario population. Sensitivity, specificity and AUC values approach those seen in validation studies of instruments like the CIDI4. We observed that with a threshold value of 4, the prevalence of AMD exceeded the prevalence of treated AMD in our sample, but was almost identical to current prevalence estimates of AMD seen in Canada using the CIDI23 and in other countries28. Additionally, prevalence by age and gender was nearly identical to that seen with instruments like the CIDI23,28.
While it is clear that the GHQ12 and other screening instruments cannot be substituted for clinical diagnosis when dealing with individuals, our data and other recent studies suggest that the GHQ12 may be usefully employed to estimate the prevalence in populations of treated and untreated AMD for surveillance purposes and for investigating prevalence and risk factors for those conditions18. Many studies have found that the instrument performs well as a screening instrument for these conditions in clinical samples8,11-13. Often, the instrument is employed to provide a more general measure of psychiatric distress9 or for screening purposes in research or clinical settings to identify candidates for more comprehensive assessments11. The way in which the instrument is scored, however, also allows the exploration of the value of higher scores as indicators or estimators of AMD, and the evidence presented here suggests that use of the GHQ12 in this way to allow epidemiologic study of AMD is feasible. Thus, these results suggest that more detailed and extensive tracking of the prevalence of AMD in Canada than has been possible in the past could be feasible, including the ability to assess the impact of societal-level factors such as changes in employment rates. Further research to confirm the ability of the GHQ12 to provide estimates of AMD prevalence, by directly comparing its performance to the CIDI, and similar diagnostic instruments, would be a valuable contribution to the epidemiological study of these disorders.
1. Goldbloom D. Psychiatric clinical skills. Burlington, MA: Elsevier; 2006. [ Links ]
2. Öhman L, Bergdahl J, Nyberg L, Nilson L-G. Longitudinal analysis of the relation between moderate long-term stress and health. Stress Health 2007; 23: 131-138. [ Links ]
3. Patten SB, Bilsker D, Goldner E. The evolving understanding of major depression epidemiology: Implications for practice and policy. Can J Psychiatry 2008; 53: 689-695. [ Links ]
4. Kessler RC, Andrews G, Mroczek D, Ustun TB, Wittchen H-U. The World Health Organization Composite International Diagnostic Interview Short Form (CIDI-SF). Int J Methods Psychiatr Res 1998; 7: 171-185. [ Links ]
5. Cairney J, Veldhuizen S, Wade T J, Kurdyak P, Streiner DL. Evaluation of 2 measures of psychological distress as screeners for depression in the general population. Can J Psychiatry 2007; 52: 111-120. [ Links ]
6. Kessler RC, Adler L, Ames M, Demler O, Faraone S, Hiripi E, et al. The World Health Organization adult ADHD self-report scale (ASRS): A short screening scale for use in the general population. Psychol Med 2005; 35: 245-256. [ Links ]
7. Goldberg DP, Rickels K, Downing R, Hesbacher P. A comparison of two psychiatric screening tests. Br J Psychiatry 1976;129:61-67. [ Links ]
8. Goldberg DP, Gater R, Sartorius N, Ustun TB, Piccinelli M, Gureje O, et al. The validity of two versions of the GHQ in the WHO study of mental illness in general health care. Psychol Med 1997; 27: 191-197. [ Links ]
9. Mann RE, Asbridge M, Stoduto G, Smart RG, Goldbloom D, Vingilis ER, et al. Psychological distress and collision involvement among adult drivers. Stress Health 2010; 26: 127-134. [ Links ]
10. Stoduto G, Dill P, Mann RE, Wells-Parker E, To-neatto T, Shuggi R. Examining the link between drinking driving and depressed mood. J Stud Alcohol Drugs 2008; 69: 777-780. [ Links ]
11. Harter M, Woll S, Wunsch A, Bengel J, Reuter K. Screening for mental disorders in cancer, cardiovascular and musculoskeletal diseases: comparison of HADS and GHQ12. Soc Psychiatry Psychiatr Epidemiol 2006; 41: 56-62. [ Links ]
12. Lewis G, Sharp D, Bartholomew J, Pelosi AJ. Computerized assessment of common mental disorders in primary care: Effect on clinical outcome. Fam Pract 1996; 13: 120-126. [ Links ]
13. Patel V, Araya R, Chowdhary N, King M, Kirkwood B, Nayak S, et al. Detecting common mental disorders in primary care in India: A comparison of five screening questionnaires. Psychol Med 2008; 38: 221-228. [ Links ]
14. Doi Y, Minowa M. Factor structure of the 12-item General Health Questionnaire in the Japanese general adult population. Psychiatry Clin Neurosci 2003; 57: 379-383. [ Links ]
15. Beard JR, Heathcote K, Brooks R, Earnest A, Kelly B. Predictors of mental disorders and their outcome in a community based cohort. Soc Psychiatry Psychiatr Epidemiol 2007; 42: 623-630. [ Links ]
16. Furukawa TA, Kessler RC, Slade T, Andrews G. The performance of the K6 and K10 screening scales for psychological distress in the Australian National Survey of Mental Health and Well-Being. Psychol Med 2003; 33: 357-362. [ Links ]
17. Donath S. The validity of the 12-item General Health Questionnaire in Australia: A comparison between three scoring methods. Austral New Zeal J Psychiatry 2001; 35: 231-235. [ Links ]
18. Bijl RV, Van Zessen G, Ravelli A, De Rilk C, Langendoen Y. The Netherlands Mental Health Survey and Incidence Study (NEMESIS): Objectives and design. Soc Psychiatry Psychiatr Epidemiol 1998; 33: 581-586. [ Links ]
19. Aday LA. Designing and conducting health surveys: A comprehensive guide (2nd ed). San Francisco: Jossey-Bass Publ.; 1996. [ Links ]
20. Ialomiteanu A, Adlaf EM. CAMH Monitor: Technical guide 2004 [Internet]. Toronto: Centre for Addiction and Mental Health; 2005 (cited 2009 June 20). Available from: http://www.camh.net/Research/Areas_of_research/Population_Life_Course_Studies/CAMH_Monitor/CM2004_TechDoc.pdf [ Links ]
21. Patten SB. Major depression prevalence is very high, but the syndrome is a poor proxy for community populations' clinical treatment needs. Can J Psychiatry 2008; 53: 411-419. [ Links ]
22. Akobeng AK. Understanding diagnostic tests 3: Receiver operating characteristic curves. Acta Paediatr 2007; 96: 644-647. [ Links ]
23. Rush B, Urbanoski K, Bassani D, Castel S, Wild TC, Strike C, et al. Prevalence of co-occurring substance use and other mental disorders in the Canadian population. Can J Psychiatry 2008; 53: 800-809. [ Links ]
24. Harrison E, Haaga J, Richards T. Self-reported drug use data: What do they reveal? Amer J Drug Alcohol Abuse 1993; 19: 423-441. [ Links ]
25. Turner C, Lessler J, Gfroefer J. Survey measurement of drug use: Methodological studies. Washington: US Government Printing Office; 1992. [ Links ]
26. Statistics Canada. Residential telephone survey 2007. Ottawa: Statistics Canada, Special Surveys Division; 2008; Cat. 56M0001XCB, www.statcan.gc.ca/bsolc/olc-cel/olc-cel?catno=56M0001X&lang=eng. [ Links ]
27. Komiti A, Jackson HJ, Judd FK, Cockram AM, Kyrios M, Yeatman R, et al. A comparison of the Composite International Diagnostic Interview (CIDI-Auto) with clinical assessment in diagnosing mood and anxiety disorders. Aust N Z J Psychiatry 2001; 35: 224-230. [ Links ]
28. Bijl RV, Ravelli A, Van Zessen G. Prevalence of psychiatric disorder in the general population: Results of the Netherlands Mental Health Survey and Incidence Study (NEMESIS). Soc Psychiatry Psychiatr Epidemiol 1998; 33: 587-595. [ Links ]
Robert E. Mann
Centre for Addiction and Mental Health, 33
Russell Street, Toronto, Ontario, M5S 2S1, Canada
Telephone: (416) 535-8501 ext. 4496
Fax: (416) 595-6899
Received: 3 February 2010
Revised: 13 October 2010
Accepted: 19 October 2010