Psychometric properties of the Swedish version of the Outcome Questionnaire-45 as administered by automated technique in a large sample of mental ill-health patients in Primary Health Care

 

 

Catharina Strid, PhD student*; Lars-Gunnar Lundh, Professor*; Claes Andersson, PhD**; Agneta Öjehagen, Professor***

* Department of Psychology, Lund University. Sweden
** Department of Criminology, Malmö University. Sweden
*** Department of Clinical Sciences, Lund, Division of Psychiatry, Lund University. Sweden

Correspondence

 

 

---

ABSTRACT

Background and Objectives: Mental health problems are common in Primary Health care and clinicians require valid and reliable instruments to make good treatment plans for these patients. The Outcome Questionnaire-45 (OQ-45) was developed by Lambert and colleagues in order to help improve outcomes of treatment.
The aims of this study were to examine the psychometric properties of the Swedish version of OQ-45 when administered by an automated technique and to evaluate if the OQ-45 uniquely could contribute to the description of the patients’ difficulties and needs beyond demographic characteristics and other instruments.
Methods: The study comprised 816 patients with mental ill-health taking part in a large randomized controlled trial, Regassa.
The OQ-45 data were collected by Interactive Voice Response IVR, a computerized, automated telephone technique. The OQ-45 consists of 45 items summarized in a total score and in three subscales; Symptoms of distress, SD, Interpersonal relations, IR, and Social role functioning, SR. Depression was measured by MADRS and health-related quality of life by EQ-5D.
Results: The OQ-45 total score showed good psychometric properties, but there was little support for its three factor structure. The OQ-45 significantly predicted level of depression and health-related quality of life in patients.
Conclusions: The OQ-45 contributed uniquely to the description of the patients´ problems. The results indicate that the total score of the Swedish version of OQ-45 can be valuable to use for clinicians in the field.

Key words: Outcome Questionnaire-45; Psychometric properties; Mental ill-health; Primary Health Care; Depression; EQ-5D.

---

 

Introduction

Mental ill-health is common in Primary Health care, 25-30% of the patients having mental health problems and some of them are not getting the treatment they need^1,2,3. It is important for clinicians to use reliable and validated instruments to make good decisions about the patient's treatment. Automated techniques can be an effective and less cost consuming way of administering different measures and to follow large samples of patients^4,5,6. Regular uses of patient-reported outcome measures can provide an efficient way of improving the assessments, processes and outcomes of health care^7,8.

The Outcome Questionnaire-45 (OQ-45) was developed by Lambert and colleagues^9,10 to improve the results of psychotherapy by tracking the treatment response during treatment. These process measures have been found useful to decrease the risk for treatment failure and to improve positive outcomes^10,11,12,13. OQ-45 has shown good test-reliability, sensitivity, internal consistency and validity in different studies^14,15 and has been translated into several languages including Swedish¹⁶. Wennberg and co-workers concluded that the psychometric properties of the Swedish version of the OQ-45 in a substances use disorders sample were acceptable except for a low internal consistency of the Social function subscale. This was in accordance with other cross-cultural studies^17,18.

Our interest was to further evaluate the OQ-45 in a large sample of primary health care patients with mental ill-health problems. The purpose of this study was to examine the psychometric properties of the Swedish version of OQ-45 administered by automated telephone technique and to examine its ability to predict level of depression and health-related quality of life in primary health care patients, beyond demographic characteristics and other standard instruments. Thereby we aimed to investigate whether OQ-45 could make any unique contribution to the description of the patients' difficulties and needs.

 

Methods

Study design

The present study was carried out within Regassa, a Swedish national multicenter Randomized Controlled Trial (RCT), which aims to study the effects of Internet based Cognitive Behavioral Therapy, ICBT, and physical activity compared to treatment as usual on sick-leave and work-ability in patients with mild to moderate depression, anxiety and stress related mental ill-health. The Regassa project took place in six different health care regions in Sweden, during 2011 to 2014, and twenty primary health care units were engaged. The project offered three months intervention and patients were followed up 3 and 12 months after baseline. Additionally the patients were followed during and after treatment through an automated telephone technique, Interactive Voice Response (IVR). The regional ethical review board in Stockholm approved the study.

 

Selection and description of participants

The patients in Regassa were included by a central research unit in Stockholm and different Primary Health care units. The study was based on voluntariness, informed consent and confidential treatment of data. After the patient had signed the agreement to take part in the study, they answered a battery of different scales and questionnaires administrated by a research assistant. At this point they registered in IVR. Thereafter they were randomized to the three treatment alternatives.

IVR

IVR is a computerized, automated telephone system which is programmed to handle questionnaires to be responded to by a touchtone telephone. At baseline the patients called the system and registered their personal telephone number and answered the questionnaires included. The computerized system was programmed to perform six follow-up assessments to capture the patients' view after one, two and three months in treatment and thereafter another three times until 12 months after baseline. At each follow-up the system called the patients' registered phone number and the patient answered the questionnaires. The results will be reported in a forthcoming paper.

Inclusion of patients

In Regassa 946 persons, aged 18-67, 689 women and 257 men, with mild to moderate depression, anxiety disorders and stress related mental health problems were included. The inclusion criteria were symptoms of depression, defined by ≥ 10 points on the Patient Health Questionnaire (PHQ9) and good knowledge in the Swedish language. Knowledge of Swedish was necessary for participation in the Internet based CBT program which was only available in Swedish. Exclusion criteria were alcohol and substance use disorders, need for psychiatric specialist care and severe somatic illness that motivated sick-leave per se.

Of the 946 patients, 816 completed all the 55 questions in the IVR system at baseline, and were included in the present study. The 130 patients who did not answer all 55 questions were regarded as drop-outs. Seventy-nine of them had answered only a few questions and the other 51 patients had not registered in IVR at all. When comparing the study group of 816 patients with the 130 drop-outs there were no significant differences regarding gender, age, education level, civil state, employment, depression (MADRS) and health-related quality of life (EQ-5D). The patients' characteristics are shown in table 1.

 

Instruments

Questionnaires in IVR

Three instruments were used in the IVR system: OQ-45, the four- item version of the Perceived Stress Scale, (PSS-4)¹⁹, and the Karolinska Sleep Questionnaire (KSQ), which contains of six questions about sleep patterns. The KSQ was not used in this study.

The OQ-45 measures the patients' level of functioning and consists of three subscales essential to define outcome: Symptoms of distress (SD), Interpersonal relations (IR) and Social role functioning (SR). The 45 items have five response alternatives: never (0), rarely (1), sometimes (2), frequently (3) and almost always (4). The SD subscale consists of 25 items, the IR subscale of 11 items and the SR of 9 items. Possible scores on the scale range from 0-180 points where 180 is the maximum negative score. The scores are related to three different levels of functioning: High 0-63, medium 64-84 and low 85-180 points²⁰. The cut-off point between clinical and normal population as established by normative data from the US is 64 points²¹.

The Perceived Stress Scale (PSS) has been developed in three different versions. The PSS 14-items scale constructed by Cohen is translated into several languages and has shown good psychometric properties in different studies^19,22,23. The Swedish version of the 10-items scale has shown good validity and internal consistency²⁴. The PSS 4-items version has shown acceptable psychometric properties in different studies and has been recommended for telephone follow-ups^19,25,26.

Other questionnaires at baseline

The Montgomery Åsberg Depression Rating Scale (MADRS) consists of 10 items with six response alternatives and has shown good psychometric properties^27,28. The EuroQol 5D (EQ-5D) is a self-report questionnaire for the measurement of health-related quality of life and consists of 5 items describing five dimensions of health with three levels in each dimension^29,30. The Mini-International Neuropsychiatric Interview (MINI) was used for diagnostics³¹. Further the patients answered a battery of socio demogra-phic questions previously used in a longitudinal population based study in Stockholm, the PART study³².

 

Statistics

All the analyses were done in SPSS 20.0. The comparison between groups on scales and age was made with independent sample t-test and for proportions the Chi-square test was used. Internal consistency was measured by Cronbach's alpha and the factor structure of OQ-45 was studied by Principal Component Analysis (PCA). To predict the level of depression (MADRS) and health related quality of life (EQ-5D) by OQ-45, a hierarchical linear regression was used.

 

Results

Table 1 shows the results on the different measures for the total group and for males and females separately. The proportion of woman was high, 74%. The education level for the total group was high; two-thirds had a post gymnasium or university education. Most of the patients were working or studying and only 5% was on sick-leave. According to their scores on OQ-45, half of the patients had a low level of functioning. The scores on MADRS showed that more than half of the patients had a moderate level of depression and only a few were severely depressed. The total score on EQ-5D was 0.55 which is in level with the mentally ill patients in a National rehabilitation program³³. Three quarters of the patients had both depression and anxiety diagnoses, i.e. the comorbidity was high. Women and men differed on three measures: educational level, OQ-45 and OQ-45 symptoms subscale.

The correlations between OQ-45 and the other included psychometric measures are strong. The results are shown in table 2.

 

The internal consistency was good for the total OQ-45, α = 0.88, and for the SD subscale, α = 0.83. The internal consistency was satisfactory for the IR subscale α = 0.75 and for the SR scale, α = 0.70. The principal component analysis with oblique rotation showed good loadings for the SD subscale but less for the IR and SR subscales. The extraction of factors resulted in 11 factors using the Kaiser criterion eigenvalues > 1 and 3-5 factors using the scree plot. When the number of factors was set to three, 15 of the 25 SD subscale items loaded > 0.40 on factor 1. Three SD items about somatic problems loaded > 0.40 on factor 2. Five IR items out of 11 loaded > 0.40 on factor 1, one IR item loaded negatively -0.42 on factor 2 and one IR item loaded > 0.40 on factor 3. Four of the total 9 SR items loaded > 0.40 on factor 1, one SR item loaded > 0.40 on factor 2 and finally two SR items loaded > 0.40 on factor 3.

Table 3 shows the results of the hierarchical multiple regression analysis to predict the level of depression using different relevant measures to explain the variance in step 1 and the contribution of the OQ-45 in step 2. The choice of measures to put into the model was based on correlations with the dependent variable, MADRS. The model explained 27% of the variance in the MADRS and the OQ-45 contributed uniquely with 8% which was statistically significant. The EQ-5D also contributed significantly to the prediction of scores on the MADRS.

 

In table 4 the results of a similar hierarchical multiple regression analysis with the EQ-5D as dependent variable is shown. The model explained 24% of the variance in the EQ-5D and the contribution of the OQ-45 was 5% which was statistically significant. The MADRS also contributed significantly to the prediction of scores on the EQ-5D.

 

The interaction with gender did not alter the picture with regard to the results of OQ-45 as a predictor.

 

Discussion

We found that the psychometric properties of the Swedish version of the OQ-45 administered by automated telephone technique in a large clinical sample were good with regard to the total OQ-45 score. However the results for the different subscales were weaker which is in accordance with results of other studies^17,18,34,35. Further the total score of the OQ-45 was found to predict level of depression and health-related quality of life in patients, beyond demographic characteristics and other instruments. Similar to other studies in Europe and USA the results indicate that the total score of the OQ-45 can be of valuable use for clinicians and researcher in the field^17,37,38. The regression analyses showed that the OQ-45 contributed uniquely to describe the patients' problems. This is probably due to the fact that the OQ-45 is a comprehensive patient reported outcome measure that includes both the patients' symptoms, interpersonal problems and social functioning. Comprehensive scales are essential for assessment, treatment planning and measuring outcome and the implications are that OQ-45 seems to be a useful tool. However separate use of the different subscales of the OQ-45 cannot be recommended on the basis of the present results.

The depression scale, MADRS, was also found to contribute uniquely to the prediction of health-related quality of life and the EQ-5D likewise contributed uniquely to the prediction of degree of depression. The latter is interesting in view of the brief nature of this instrument as it shows the usefulness of the EQ-5D as a patient reported outcome measure in clinical settings.

The large clinical sample of Primary Health Care Patients is an advantage of the study and the technique of data collecting of OQ-45 has to our knowledge not been used before. A limitation is however that the sample is not a random sample of Primary Health care patients. On the other hand, the sample seems representative both in terms of gender and education. The large proportion of women reflects the gender distribution in the Primary health Care in Sweden³⁹ and other European countries^40,41. Also in terms of education level the sample is similar to a large Swedish cohort study in the Primary Health Care³⁹.

The automated data collecting technique of OQ-45 showed results similar to other samples where the data collection was made by paper and pencil^11,16,36. These results are promising for both the use of OQ-45 total score and for the automated data collecting technique. Further evaluation of administering OQ-45 by automated technique will nevertheless be important in order to generalize these findings. Forthcoming studies using Regassa follow-up data may contribute to this.

 

Conflict of interest

There is no conflict of interest to declare.

 

References

1. Nordström A, Bodlund O. Every third patient in primary care suffers from depression, anxiety or alcohol problems. Nordic J Psychiatry. 2008; 62 (3): 250-255.         [ Links ]

2. Cravern MA, Bland R. Depression in Primary care: Current and future challenges. Can J Psychiatry. 2013; 58 (8): 442-448.         [ Links ]

3. McHugh P, Brennen J, Galligan N, McGonagle C, Byrne M. Evaluation of a primary care adult mental health service: Year 2. Ment Health Fam Med. 2013; 10: 53-59.         [ Links ]

4. Andersson C, Danielsson S, Silfverberg-Dymling G, Löndahl G, Johansson BA. Evaluation of Interactive Voice Response (IVR) and postal survey in follow-up of children and adolescents discharged from psychiatric outpatient treatment: A randomized controlled trial. Springerplus. 2014; 3: 77.         [ Links ]

5. Corkrey R, Parkinson L. Interactive voice response: Review of studies 1989-2000. Behav Res Methods, Instrum & Computers. 2002; 34 (3): 342-353.         [ Links ]

6. Naylor MR, Naud S, Keefe FJ, Helzer JE. Therapeutic Interactive Voice Response (TIVR) to reduce analgesic medication use for chronic pain management. J Pain. 2010; 11 (12): 1410-1419.         [ Links ]

7. Marshall S, Haywood K, Fitzpatrick R. Impact of patient-reported outcome measures on routine practice: A structured review. J Eval Clin Pract. 2005; 12 (5): 559-568.         [ Links ]

8. Lambert MJ, Hansen NB, Finch AE. Patient-Focused research: Using patient outcome data to enhance treatment effects. J Cons Clin Psychol. 2001; 69 (2): 159-172.         [ Links ]

9. Lambert MJ. The efficacy and effectiveness of psychotherapy. In Lambert MJ eds Bergin and Gartfield's handbook of psychotherapy and behavior change. 6^th edition. New York: Wiley. 2014; 169-218.         [ Links ]

10. Slade K, Whipple JL. Improving outcomes for poorly responding clients: The use of clinical support tools and feedback to clients. J Clin Psychol. 2005; In session 61: 175-185.         [ Links ]

11. Hansson H, Rundberg J, Österling A, Öjehagen A, Berglund M. Intervention with feedback using Outcome Quetsionnaire 45 (OQ-45) in a Swedish psychiatric outpatient population: A randomized controlled trial. Nordic J Psychiatry. 2012; Early Online: 1-8.         [ Links ]

12. Lambert MJ. Presidential address: What we have learned from a decade of research aimed at improving psychotherapy outcome in routine care. Psychother Res. 2007; 17 (1): 1-14.         [ Links ]

13. Lambert MJ. Outcome in psychotherapy: The past and importance advances. Psychother. 2013; 50 (1): 42-51.         [ Links ]

14. Lambert MJ, Burlingame GM, Umphress V, Hansen NB, Vermeersch DA, Clouse GC, et al. The reliability and validity of the Outcome Questionnaire. Clin Psychol Psychother. 1996; 3 (4): 249-258.         [ Links ]

15. Vermeersch DA, Whipple JL, Lambert MJ, Hawkins J, Burchfield CM, Okiishi JC. Outcome questionnaire: Is it sensitive to changes in counseling center clients? J Couns Psychol. 2005; 51 (1): 38-49.         [ Links ]

16. Wennberg P, Philips B, de Jong K. The Swedish version of the Outcome Questionnaire (OQ-45): Reliability and factor structure in a substance abuse sample. Psychol Psychother: Theory, Res Pract. 2010; 83 (3): 325-329.         [ Links ]

17. De Jong K, Nugter MA, Polak MG, Wagenborg JEA, Spinhoven P, Heiser WJ. The Outcome Questionnaire (OQ-45) in a Dutch population: A cross-cultural validation. Clin Psychol Psychother. 2007; 14: 288-301.         [ Links ]

18. Lo Coco G, Chiappelli M, Bensi L, Gullo S, Prestano C, Lambert MJ. The factorial structure of the Outcome Questionnaire-45: A study with an Italian sample. Clin Psychol Psychother. 2008; 15: 418-423.         [ Links ]

19. Cohen S, Kamarack T, Mermelstein R. A global measure of perceived stress. J Health Soc Behav. 1983; 24 (12): 385-396.         [ Links ]

20. Lambert MJ, Morton JJ, Hatfield D, Harmon C, Hamilton S, Reid RC, et al. Administration and scoring manual for the Outcome Questionnaire - 45. Orem. UT: Am Prof Credential Serv 2004. 2004.         [ Links ]

21. Andersson EM, Lambert MJ. A survival analysis of clinically significant change in outpatient psychotherapy. J Clin Psychol. 2001; 57 (7): 875-888.         [ Links ]

22. Eun-Hyan L. Review of the psychometric evidence of the Perceived stress scale. Asian Nurs Res. 2012; 6: 121-127.         [ Links ]

23. Eklund M, Bäckström M, Tuvesson H. Psychometric properties and factor structure of the Swedish version of the Perceived stress scale. Nord J Psychiatry. 2014. Early Online.         [ Links ]

24. Nordin M, Nordin S. Psychometric evaluation and normative data of the Swedish version of the 10-item Perceived stress scale. Scand J Psychol. 2013; 54 (6): 502-507.         [ Links ]

25. Warttig SL, Forshaw MJ, South J, White AK. New, normative, English-sample data for the Short Form Perceived Stress Scale (PSS-4). J Health Psychol. 2013; 18 (12): 1617-1628.         [ Links ]

26. Karam F, Berard A, Sheehy O, Huneau M-C, Briggs G, Chambers C, et al. Reliability and validity of the 4-item Perceived stress scale among pregnant women: Results from the OTIS depressants study. Res Nurs Health. 2012; 35 (4): 363-375.         [ Links ]

27. Montgomery S, Åsberg M. A new depression scale designed to be sensitive to change. Br J Psychiatry. 1979; 134 (4): 382-389.         [ Links ]

28. Quilty LC, Robinson JJ, Rolland JP, Fruyt FD, Rouillon F, Bagby RM. The structure of the Montgomery Åsberg depression rating scale over the course of treatment for depression. Int J Methods Psychiatr Res. 2013; 22 (3): 175-184.         [ Links ]

29. Dolan P. Modeling valuations for EuroQol health states. Med Care. 1997; 35 (11): 1095-1108.         [ Links ]

30. Greiner W, Wijnen T, Nieuwenhuizen M, Oppe S, Badia X, Busschblad J, et al. A single European currency for EQ-5D health states: Results from a six-country study. Eur J Health Econ. 2003; 4 (3): 222-231.         [ Links ]

31. Sheehan DV, Lecrubier Y, Harnett-Sheehan K, Amorim P, Janavs J, Weiller E, et al. The Mini International Neuropsychiatric Interview (M.I.N.I.). J Clin Psychiatry. 1998; 59 (20): 22-33.         [ Links ]

32. Hällström T, Damström-Thakker K, Forsell Y, Tinghög P, Lundberg I. The PARTstudy, technical report. 2004. http://www.folkhalsoguiden.se/Rapport.aspx?id=1136        [ Links ]

33. Busch H, Bonnevier H, Hagberg J, Karlsson LM, Bodin L, Norlund A. En nationell utvärdering av rehabiliteringsgarantins effekter på sjukfrånvaro och hälsa. (A national evaluation of the effects of a rehabilitation programme on sick-leave and health) Slutrapport, del l. 2011; Enheten för interventions- och implementeringsforskning, Institutet för miljömedicin (IMM) Karolinska Institutet, Stockholm Swedish.         [ Links ]

34. Mueller R, Lambert MJ, Burlingame GM. The Outcome Questionnaire: A confirmatory factor analysis. J Personal Assess. 1998; 70: 248-262.         [ Links ]

35. Chao RC-L, Green KE. Rasch analysis of the Outcome Questionnaire with African Americans. Psychol Assess. 2013; 25 (2): 568-582.         [ Links ]

36. Rundberg J, Hansson H, Öjehagen A, Berglund M. Feedback för förändring: Tillämpning inom primärvården. (Feedback for change: Application in the primary health care) Institutionen Kliniska vetenskaper, Lund, Avdelningen psykiatri, Institutionen Hälsa, vård och samhälle, Kliniskt centrum för hälsofrämjande vård, Lunds universitet Swedish.         [ Links ]

37. Amble I, Gude T, Stubdal S, Oktedalen T, Skjorten AM, Andersen BJ, et al. Psychometric proporties of the Outcome Questionnaire-45.2: The Norwegian version in an international context. Psychother. Res. 2014; 24 (4): 504-513.         [ Links ]

38. Thalmayer AG. Alternative models of the Outcome Quetsionnaire-45. Eur J Psychol Assess. Online first publication, july 2014.         [ Links ]

39. Lejtzèn N, Sundquist J, Sundquist K, Li X. Depression and anxiety in Swedish primary health care: Prevalence, incidence and risk factors. Eur Arch Psychiatry Clin Neurosci. 2014; 264: 235-45.         [ Links ]

40. Roca M, Gili M, Garcia-Garcia M, Salva J, Vives M, Garcia Campayo J, et al. Prevalence and comorbidity of common mental disorders in primary care. J Affect Disord. 2009; 119 (1-3): 52-58.         [ Links ]

41. Andriopoulos P, Lotti-Lykousa M, Pappa E, Papadopoulos AA, Niakas D. Depression, quality of life and primary care: A cross-sectional study. J Epidemiol Glob Health. 2013; 3: 245-252.         [ Links ]

 

 

Correspondence:
Catharina Strid
Lund University
Department of Psychology
Box 213
S-221 00 Lund. Sweden
Tel: +46763455313
E-mail: catharina.strid@psy.lu.se

Received: 3 July 2014
Revised: 14 August 2014
Accepted: 8 September 2014