Suicide risk in a Portuguese non-clinical sample of adults

 

 

Rui C. Campos*; Ronald R. Holden**

* Department of Psychology and Research Center on Education and Psychology (CIEP), University of Évora, Portugal
** Ronald R. Holden, Department of Psychology, Queen’s University, Canada.

Correspondence

 

 

---

ABSTRACT

Background and Objectives: This exploratory study simultaneously tests the contribution of socio-demographic, clinical, distress, and personality variables for identifying suicide risk in a non-clinical sample.
Methods: A convenience sample of 810 adults ranging in age from 19 to 67 years (M = 36.34, SD = 12.46) and living in various Portuguese regions participated. Their education varied from 6 to 21 years of schooling (M = 11.74, SD = 5.14). Participants responded to socio-demographic questions, the Depressive Experiences Questionnaire, the Center for the Epidemiological Studies of Depression Scale, and the Suicide Behaviors Questionnaire-Revised.
Results: In the present sample, 4.3% of participants indicated past suicide attempts, 27.9% reported some lifetime suicide ideation, and 6.4% indicated a past suicide plan. Depressive symptoms, having seen a psychologist or psychiatrist, self-criticism, and education discriminated between participants who had attempted suicide (n = 35) and those who had not attempted to die by suicide (n = 775). Depressive symptoms, having seen a psychologist or psychiatrist, self-criticism, psychiatric disease and age discriminated between participants who scored below (n = 650) and who scored equal to or above (n = 160) the cut-off score for the Suicide Behaviors Questionnaire-Revised.
Conclusions: Results have implications for the assessment of suicide risk.

Key words: Suicide risk; Non-clinical sample; Socio-demographic variables; Clinical variables; Self-criticism; Depressive symptoms.

---

 

Introduction

Suicide is a major mental health issue throughout the human life span in both clinical and non-clinical populations. The World Health Organization¹ estimates that each year almost one million people die by suicide worldwide. In Portugal, more than a thousand people die by suicide every year (10.3 per 100,000 in the population), with suicide being the leading cause of unnatural death². Non-fatal suicidal behaviors, such as suicide attempts, have estimated rates that are 20 or more times greater than completed suicides³. Given these statistics, understanding the risk factors for suicide becomes a public health imperative. There is a range of theoretical models for suicidal behavior which have included socio-cultural and psychological perspectives⁴ demonstrating the potential influence of various risk factors at several levels, such as individual and psychosocial factors^5,6.

Demographic variables have been linked to suicide and they can provide a preliminary assessment of suicide risk⁷. Males are often viewed as the sex at greatest risk for suicide⁷. Although females are more likely than males to attempt suicide⁸ males are more likely than females to die by suicide⁹. Being elderly has also been associated with suicide⁹. Although adolescence is a high-risk period for suicide attempts, individuals over 50 years of age are more likely to die by suicide than are young people¹⁰. Elevated suicide risk has also been associated with being widowed¹¹ separated, or divorced^10,12. Additionally, being unmarried is seen as related to suicide risk^7,13,14. Furthermore, unemployment is another risk factor for suicide^{14,15,16,17,18} with financial problems related to unemployment and low educational level being indicated as relevant by persons who have attempted suicide¹⁹.

To understand suicide, it is useful to describe the broader societal framework within which the phenomenon takes place. Socio-cultural traditions influence and constrain how individuals construct meaning about important social events²⁰. In Portugal, Alentejo is the region with the highest rate of suicide². It is unclear why this regional difference occurs, but it certainly constitutes an endemic phenomenon.

Several clinical variables have also been related to suicide risk. The presence of a chronic disease is considered a risk factor²¹ and distress, particularly depression, is also indicated as a powerful predictor of suicidal behaviors^22,23,24. Indeed, a psychiatric diagnosis is one of the most important predictors of eventual suicide⁷ with the majority of individuals who have died by suicide meeting the criteria for a psychiatric disorder²⁵. The presence of a psychiatric disorder is also related to having seen a mental health professional.

A number of dysfunctional personality traits have also been linked to suicidality²⁶. Substantial clinical²⁷ and empirical^22,28 evidence indicates that the personality traits of self-critical perfectionism play a major role in suicide, especially in adolescence/early adulthood²⁹ and that self-critical individuals are at elevated risk for serious suicide attempts²⁷.

The current exploratory study aims to simultaneously evaluate the contribution of several variables deemed important in the literature to suicide risk in a large Portuguese non-clinical sample of adults. The main goal is to test the importance of these variables in the prediction of suicide risk. We measured two indicators of suicide risk, both the presence of a past suicide attempt and the scores for a well validated screening psychometric measure of suicide risk, the Suicide Behaviors Questionnaire-Revised (SBQ-R). As predictors, we used a personality variable, self-criticism, a distress variable, depressive symptoms, six socio-demographic variables, gender, age, education level, region of residence, employment status and civil status, and three clinical variables, having a chronic disease that was not a psychiatric disorder, having visited a psychologist or psychiatrist, and the presence of a psychiatric disorder. We expected that these 11 predictors would discriminate between community participants who had previously attempted suicide and those who had not, and between participants who scored below and those scoring equal to or above the risk-identified cut-off score for the SBQ-R.

 

Methods

Participants and Procedure

A convenience sample of 810 adults (403 men, 407 women), ranging in age from 19 to 67 years (M = 36.34, SD = 12.46) and living in several Portuguese regions, mostly in Alentejo, participated in this study. Their education ranged from 6 to 21 years of schooling (M = 11.74, SD = 5.14). A minority (19.1%) was unemployed and 49.5% was married or living with a romantic partner (see Table 1). Participants were contacted directly in public places and asked to participate in a study concerning personality and mood. Of the 966 individuals initially contacted, 108 declined to participate. Protocols of 48 individuals initially interviewed were eliminated due to missing socio-demographic information or an elevated number of missing responses, thus yielding a final sample of 810 participants who volunteered to participate after signing an informed consent form. Participants were not compensated for their participation. All protocols contained a socio-demographic questionnaire and three psychometric measures to assess depressive symptoms, suicide risk, and self-critical personality traits. Protocols were collected individually by trained research assistants and instructions were provided in written form. The three psychometric questionnaires were presented in a counterbalanced order, but the socio-demographic questionnaire was always the first to be answered. Data collection fulfilled all recommended ethical guidelines of the Portuguese Psychologists Board and of the American Psychological Association. Participants were given the opportunity to discontinue their participation at any time.

 

Measures

Sociodemographic questionnaire. Participants provided socio-demographic information concerning gender, age, education, civil status, employment, and district / region of residence. Participants were also asked about three clinically relevant variables: whether they had a chronic illness, whether they had ever seen a psychologist or psychiatrist, and whether they had ever been diagnosed with a psychiatric disorder. Data for the socio-demographic and clinical variables appear in Table 1.

Center for the Epidemiological Studies of Depression Scale (CES-D). This 20-item inventory (CES-D)³⁰ was used to assess depressive symptoms. Individuals are asked to indicate the frequency with which they have experienced each of 20 symptoms of depression over the past week on a 4-point scale (0 to 3). The CES-D has adequate psychometric properties³¹. The CES-D was adapted for the Portuguese population by Gonçalves and Fagulha³² and has adequate psychometric characteristics with Cronbach's alpha values varying between 0.87 and 0.92 across samples. In the present study, the alpha coefficient was 0.81.

The Depressive Experiences Questionnaire (DEQ). The DEQ was used to assess self-criticism. This measure includes 66 items and yields three global scores: Dependency, Self-Criticism, and Efficacy. The first two scales are associated with anaclitic and introjective personality dimensions³³. The validity of these scales has been demonstrated in several studies, in various countries³⁴. The Portu-guese version³⁵ of the DEQ has adequate psychometric properties. The internal consistency and the factor structure are very similar to those initially obtained by Blatt et al.³³. In the present study, Cronbach's alpha for the Self-Criticism scale was 0.77.

Suicide Behaviors Questionnaire Revised. This measure (SBQ-R)³⁶ was used to assess suicide risk. The questionnaire was adapted for the Portuguese population by Campos et al.³⁷. The questionnaire is composed of four items. The first item is: "Have you ever thought about or attempted to kill yourself?" Total scores can range between 3 and 18, and in this study ranged between 3 and 16. In the present study, we used two sources of information regarding suicide risk based on the questionnaire: results of the first item (to asses if the person had previously attempted suicide) and the total score (as a validated index of suicide risk³⁶). In the current investigation, Cronbach's alpha for the scale was 0.74.

 

Data analysis

We computed the percentage of participants who reported past suicide ideation, past suicide plans and past suicide attempts based on Item 1 of the SBQ-R. Then, based on that item and on the total score of the SBQ-R, we defined two groupings of suicide risk: those having attempted suicide (n = 35) vs. those not having attempted suicide (n = 775); and those scoring equal to or above the cut-off score of the SBQ-R (n = 160) vs those scoring below the cut-off score (n = 650). In Tables 2 and 3, we present descriptive results for the socio-demographic variables, clinical variables, depressive symptoms and self-criticism as a function of suicide risk groupings. For those variables, we then compared participants who had attempted suicide with those who hadn't attempted suicide and also compared participants who scored below the cut-off score with those who scored equal to or above the cut-off score.

Finally, we evaluated predictors for their ability to provide unique information in identifying suicide risk groupings. For this, we used stepwise Discriminant Function Analysis (DFA) and Receiver Operating characteristic Curve (ROC) analysis. In stepwise DFA, predictors are successively added to a prediction equation provided that the increment in explained variance is statistically significant. Once, the DFA solution was obtained, participants' discriminant scores were saved and used for the subsequent ROC analysis. A ROC curve plots sensitivity (i.e., hit rate) against 1- specificity (i.e., false alarm rate). ROC analyses do not assume distributional normality or homos-cedasticity and are independent of prevalence. ROC information is summarized as an AUC (Area Under the Curve) that is compared to a chance value of 0.50 represented by the main diagonal of the ROC plot. The AUC is the probability that a pair of cases selected randomly from the two underlying distributions will be classified correctly and 1.00 - AUC is the overlap between the two distributions. AUC scores between 0.50 and 0.70, between 0.70 and 0.90, and above 0.90 are designated as low, moderate, and high accuracy, respectively³⁸. AUCs can be tested for statistical significance³⁹.

 

Results

The total mean value obtained in the present study for the SBQ-R was 4.94 (SD = 2.46). 20.7% of our sample obtained a value equal to or above the cutoff point of 7 proposed by Osman et al.³⁶. In the present sample 4.3% of the participants indicated past suicide attempts, 27.9% reported lifetime suicidal ideation, and 6.4% indicated a past suicide plan.

Comparison between participants who had (n = 35) and who hadn't (n = 775) attempted suicide, using discriminant function analysis.

Table 2 presents results for the studied variables for groups of participants who had attempted suicide and those who hadn't attempted to die by suicide. We computed a stepwise discriminant function analysis to compare both groups based on the studied variables^*. The final discriminant function was significant, Wilks' Lambda = 0.922, χ²(4) = 63.28, p < 0.001 and four variables contributed with unique and significant variance toward discriminating between groups. In order of entry, contributing predictors were depressive symptoms, having visited a psychologist or psychiatrist, self-criticism, and education (lower). Corresponding standardized discriminant function coefficients were 0.36 (95% CI (0.09, 0.60)) for depressive symptoms, 0.56 (95% CI (0.34, 0.82)) for having visited a psychologist or psychiatrist, 0.46 (95% CI (0.13, 0.66)) for self-criticism, and -0.30 (95% CI (-0.06, -0.47)) for education.

We then used the discriminant function scores based on these four predictors in a Receiver Operator characteristic Curve (ROC) analysis to predict attempter status. This produced a plot (Figure 1) of sensitivity vs 1-specifity that had an Area Under the Curve (AUC) of 0.843 (95% CI 0.80, 0.89), z = 35.13, p < 0.001, regarded as a moderate level of diagnostic accuracy³⁸.

 

Comparison between participants who scored below (n = 650) and those who sco-red equal to or above the cut-off point in the SBQ-R (n = 160), using discriminant function analysis.

Table 3 presents results for the studied variables for groups of participants who scored below or those who scored equal to or above the cut-off score for the SBQ-R. We computed a stepwise discriminant function analysis to compare both groups based on the studied variables. The final discriminant function was significant, Wilks' Lambda = 0.744, χ²(5) = 230.17, p < 0.001 with five variables contributing with unique and significant variance toward discriminating between groups. In order of entry, contributing predictors were depressive symptoms, having visited a psychologist or psychiatrist, self-criticism, having a psychiatric disorder, and age. Corresponding standardized discriminant function coefficients were 0.62 (95% CI (0.45, 0.74)) for depressive symptoms, 0.39 (95% CI (0.24, 0.54)) for having visited a psychologist or psychiatrist, 0.32 (95% CI (0.16, 0.47)) for self-criticism, 0.21 (95% CI (0.02, 0.37)) for having a psychiatric disorder, and 0.15 (95% CI (0.01, 0.29)) for age.

Discriminant function scores based on these five predictors were subsequently used in a Receiver Operator characteristic Curve (ROC) analysis to predict SBQ-R risk status. The obtained plot (Figure 2) of sensitivity vs 1-specifity had an Area Under the Curve (AUC) of 0.831 (95% CI (0.80, 0.87)), z = 48.89, p < 0.001, again signifying a moderate level of diagnostic accuracy³⁸.

 

Discussion

This study assessed suicide risk in a Portuguese community sample and tested whether a cluster of eleven variables (six socio-demographic variables, a distress variable, a personality variable and three clinical variables) could discriminate at-risk individuals from non-at-risk individuals, using two criteria for suicide risk: previously having attempted suicide and presenting a score equal to or above the cut-off point for a well validated screening suicide risk psychometric measure.

When we consider past suicide attempt as an indicator of suicide risk, our results indicate that the variables of depressive symptoms, having visited a psychologist or psychiatrist, self-criticism and education discriminate between suicide at-risk and non-at-risk participants. These results are in accord with previous findings reported in the literature^{19,22,24,37,40}. This implies that highly self-critical individuals, who are depressed, have a lower level of education, and probably present emotional suffering (i.e., have visited a mental health professional) are mostly like to be at risk for suicidal behaviors. In fact, self-criticism can be considered a risk factor not only for suicidality, but more generally for psychological distress and a variety of other emotional disorders³⁴. It appears that a constellation of emotional distress^22,37 and pain⁴⁰ associated with a lower education level¹⁹ (which can be associated with a low income, and low socio-economic level), may increase suicide risk.

When we consider scores equal to or above the cut-off point in a suicide risk screening measure as an indicator of suicide risk, the variables of depressive symptoms, having visited a psychologist or psychiatrist, self-criticism, a psychiatric diagnosis, and age discriminate between suicide at-risk and non-at-risk participants. These results also agree strongly with the previous reports^{7,9,10,23,25,37}. This implies that high self-critical individuals, who are older, depressed, have a psychiatric diagnosis, and who are presenting emotional suffering⁴⁰ are at risk for suicide behaviors. With this second indicator of suicide risk, again, emotional disturbance^7,25, psychological distress²² and pain⁴⁰ and mar-ked introjective³⁷ traits may be associated with an increased suicide risk. The importance of the variable age is more ambiguous, because its statistical significance in discriminating groups of suicide and non-suicide risk is marginal.

Suicidal behavior is a major public health issue with it being a cause of death every year for over 1000 persons in Portugal and a million people worldwide. However, despite its frequency, suicide is also a low base rate behavior (10.3 per 100,000 in Portugal). As such, screening for suicide risk is both an important and a challenging undertaking. Identifying relevant risk factors for suicide can provide mental health professionals with important information concerning those individuals who should be assessed more extensively for being suicidal. Across two indices of suicide risk, our present study identified depression, having visited a psychiatrist or psychologist, and self-criticism as the three most important distinguishing features. Based on this, our recommendation would be that individuals with these characteristics be referred for additional suicide risk screening. This is more strongly advised when such persons are also older and less educated.

 

Limitations, conclusions and clinical implications

The present study has some limitations. First, although we used a large sample, it was a sample of convenience and may not be representative of the Portuguese population. There may have been a selection bias in that we invited individuals to participate by contacting them in public places. Those at higher risk for suicide may be less likely to be in public places and more likely to be in their homes or residing in institutions. Second, we relied on self-report measures to collect data, namely information regarding personality and depression. Third, we did not interview participants regarding their specific psychiatric diagnoses. Fourth, we used a cross-sectional design, not a longitudinal design. Finally, the study is limited because there may be other important variables for suicide risk that were not assessed here, such as negative life events and psychache/psychological pain. Future studies should address these limitations.

However, based on the present exploratory research, several types of variables should be considered in assessing potential suicide risk. These are dysfunctional personality traits, distress (specifically depressive symptoms), socio-demographic variables (age and low education), and clinical variables (having a psychiatric diagnosis and having received professional help, which can be related to the experience of psychological pain or emotional suffering). Although our research was exploratory, our results do confirm previous findings of the importance of the predictors that we have identified. As such, we suggest that the presence of these identified variables in their clients should lead mental health professionals toward conducting more extensive, specific assessments of suicide risk.

 

Acknowledgments

We would like to acknowledge the research assistants of the University of Évora, Portugal for their valuable help with the data collection. Grateful thanks are also extended to all of the participants in this study.

 

Conflict of interest

The authors do not have any conflict of interest.

---

* To overcome potential issues regarding data distributional assumptions of normality and homoscedasticity, bootstrapping with 1,000 resamples was used for tests of significance in the discriminant function analyses.

 

 

References

1. World Health Organization Suicide. Country reports and charts available: Portugal. 2012.         [ Links ]

2. National Institute of Statistics. Óbitos por lesões provocadas intencionalmente. 2013.         [ Links ]

3. Fotti SA, Katz LY, Afifi TO, Cox BJ. The associations between peer and parental pelationships and suicidal behaviours in early adolescents. Canadian Journal of Psychiatry. 2006; 51: 698-703.         [ Links ]

4. Goldney RD, Schioldann JA. Evolution of the concept of altruistic suicide in Pre-Durkheim Suicidology. Archives of Suicide Research. 2004;8: 23-7.         [ Links ]

5. Beautrais AL, Collings SCD, Ehrhardt P, et al. Suicide prevention: A review of evidence of risk and protective factors, and points of effective intervention. Wellington, New Zealand: Ministry of Health. 2005.         [ Links ]

6. Maris RW. 'Suicide: Seminar'. The Lancet. 2002; 360: 319-26.         [ Links ]

7. Overholser JC, Braden A, Dieter L. Understanding suicide risk: Identification of high-risk groups during high-risk times. Journal of Clinical Psychology. 2012; 68: 334-348.         [ Links ]

8. Nock M, Borges G, Bromet E, Alonso J, Angermeyer M, Beautrais A, Williams D. Cross-national prevalence and risk factors for Suicidal ideation, plans, and attempts. British Journal of Psychiatry. 2008; 192: 98-105.         [ Links ]

9. Beautrais AL. Suicides and serious suicide attempts: Two populations or one? Psychological Medicine. 2001; 3: 837-845.         [ Links ]

10. Kposowa AJ. Marital status and suicide in the National Longitudinal Mortality Study. Journal of Epidemiology & Community Health. 2002; 54: 254-261.         [ Links ]

11. Borges G, Angst J, Nock MK, Ruscio A, Kessler RC. Risk factors for the incidence and persistence of suicide-related outcomes: A 10-year follow-up study using the National Comorbidity Surveys. Journal of Affective Disorders. 2006; 10: 25-33.         [ Links ]

12. Cutright P, Stack S, Fernquist RM. The age structures and marital status differences of married and not married male suicide rates: 12 developed countries. Archives of Suicide Research. 2006; 10: 365-382.         [ Links ]

13. McIntosh JL, Santos JF, Hubbard RW, Overholser JC. Elder suicide: Research, theory and treatment. Washington, D.C.: American Psychological Association. 1994.         [ Links ]

14. Qin P, Agerbo E, Mortense PB. Suicide risk in relation to socioeconomic, demographic, psychiatric, and familial factors: A national register-based study of all suicides in Denmark, 1981-1997. Archives of General Psychiatry. 2003; 160: 765-772.         [ Links ]

15. Marmot MG. Social determinants of mental health. Presentation at the International Forum on Innovation in Mental Health, Lisbon, Portugal. 2013, October.         [ Links ]

16. Mortensen PB, Agerbo E, Erikson T, Qin P, Westergaard-Nielsen N. Psychiatric illness and risk factors for suicide in Denmark. The Lancet. 2000; 355: 9-12.         [ Links ]

17. Koskinen O, Pukkila K, Hakko H, Tjihonen J, Vaisanen E, Sarkioia T, Rasanen P. Is occupation relevant in suicide? Journal of Affective Disorders. 2002; 70: 197-203.         [ Links ]

18. Platt S, Hawton K. In K Hawton, Van Heeringen, editors. The International Handbook of Suicide and Attempted Suicide. Chichester: John Wiley & Sons. 2000; 303-378.         [ Links ]

19. Hall R, Platt D, Hall R. Suicide risk assessment: A review of risk factors for suicide in 100 patients who made severe suicide attempts. Psychosomatics. 1999; 40: 18-27.         [ Links ]

20. Valsiner J. Culture and human development. London: Sage. 2000.         [ Links ]

21. Bostwick M, Cohen L. Differentiating suicide from life-ending acts and end-of life decisions: A model based on chronic kidney disease and dialysis. Psychosomatics. 2000; 50: 1-7.         [ Links ]

22. Campos RC, Besser A, Blatt SJ. Distress mediates the association between personality predispositions and suicidality: A preliminary study in a Portuguese community sample. Archives of Suicide Research. 2012; 16: 1-16.         [ Links ]

23. Lamis DA, Malone PS, Langhinrichsen-Rohling J, Elis TE. Body investment, depression, and alcohol use as risk factors for suicide proneness in college students. Crisis: The Journal of Crisis Intervention and Suicide Prevention. 2010; 31: 118-127.         [ Links ]

24. Youssef G, Plancherel B, Laget,J, Corcos M, Flament MF, Halfon O. Personality trait risk factors for attempted suicide among young women with eating disorders. European Psychiatry. 2004; 19: 131-139.         [ Links ]

25. Cavanagh JTO, Carson AJ, Sharpe M, Lawrie SM. Psychological autopsy studies of suicide: A systematic review. Psychological Medicine. 2003; 33: 395-405.         [ Links ]

26. Ash P. In RI Simon, RE Hales, editors. Textbook of suicide assessment and management. Washington, DC: American Psychiatric Publishing. 2006; 35-55.         [ Links ]

27. Blatt SJ. The destructiveness of perfectionism: Implications for the treatment of self-depression. American Psychologist. 1995; 53: 103-120.         [ Links ]

28. Blatt S, Quinlan D, Chevron E, McDonald C, Zuroff D. Dependency and self-criticism: Psychological dimensions of depression. Journal of Consulting and Clinical Psychology. 1982; 50: 113-124.         [ Links ]

29. O'Connor RC. The relations between perfectionism and suicidality: A systematic review. Suicide and Life-Threatening Behavior. 2007; 37: 698-714.         [ Links ]

30. Radloff LS. The CES-D Scale: A self-report depression scale for research in the general population. Applied Psychological Measurement. 1977; 1: 385-401.         [ Links ]

31. Eaton WW, Muntaner C, Smith C, Tien A, Ybarra M. In ME Maruish, editors. The use of psychological testing for treatment planning and outcomes assessment, 3rd ed. Mahwah, NJ: Lawrence Erlbaum. 2004; 363-377.         [ Links ]

32. Gonçalves B, Fagulha T. The Portuguese version of the Center for Epidemiologic Studies Depression Scale (CES-D). European Journal of Psychological Assessment. 2004; 20: 339-348.         [ Links ]

33. Blatt S, D'Afflitti J, Quinlan D. Experiences of depression in normal young adults. Journal of Abnormal Psychology. 1976; 85: 383-389.         [ Links ]

34. Blatt SJ. Polarities of experience: Relatedness and self-definition in personality development, psychopathology, and the therapeutic process. Washington, DC: American Psychological Association. 2008.         [ Links ]

34. Hanley JA, McNeil BJ. A method of comparing the areas under receiving operating characteristic curves derived from the same cases. Radiology. 1983; 148: 839-843.         [ Links ]

35. Campos RC. Questionário de Experiências depressivas: Manual. Évora, Portugal: Departamento de Psicologia da Universidade de Évora. 2009.         [ Links ]

36. Osman A, Bagge CL, Guitierrez PM, Konick LC, Barrios FX. The Suicidal Behaviors Questionnaire-Revised (SBQ-R): Validation with clinical and nonclinical samples. Assessment. 2001; 5: 443-454.         [ Links ]

37. Campos RC, Besser A, Blatt S.J. Recollections of parental rejection, self-criticism and depression in suicidality. Archives of Suicide Research. 2013; 17: 58-74.         [ Links ]

38. Streiner DL, Cairney J. What's under the ROC? An introduction to receiver operating characteristics curves. Canadian Journal of Psychiatry. 2007; 52: 121-128.         [ Links ]

39. Hanley JA, McNeil BJ. A method of comparing the areas under receiving operating characteristic curves derived from the same cases. Radiology. 1983; 148: 839-843.         [ Links ]

40. Holden RR, Mehta K, Cunningham EJ, McLeod LD. Development and preliminary validation of a scale of psychache. Canadian Journal of Behavioural Science. 2001; 33: 224-232.         [ Links ]

 

Correspondence:
Rui C. Campos
Departamento de Psicología
Escola de Ciências Sociais,
Universidade de Évora
Apartado 94, Évora
Portugal 7002-554
Tel: + 351 969046400
Fax: + 351266768073
E-mail: rcampos@uevora.pt

Received: 3 September 2014
Revised: 28 September 2014
Accepted: 1 October 2014