The Global Mental Health Assessment Tool Primary Care and General Health Setting Version (GMHAT/PC) - Spanish version: A validity and feasibility study

 

 

Paola Tejada^a; Luís Eduardo Jaramillo^b; Jefferson García^b and Vimal Sharma^c

^a University of Chester. United Kingdom
^b National University of Colombia. Colombia
^c Cheshire and Wirral NHS Foundation Trust and University of Chester, Chester. United Kingdom

Correspondence

 

 

---

ABSTRACT

Background and Objectives: The study aims to assess the feasibility and the level of agreement between the Spanish version GMHAT/PC diagnosis and psychiatrists'; ICD-10 based clinical diagnosis.
Methods: Participants in the study ranged from those who were in remission to others who had different mental illnesses. They were recruited from inpatient and outpatient mental health settings. All consecutive patients were interviewed using Spanish version of GMHAT/PC and they were assessed independently by psychiatrists to in order to get their ICD-10 based diagnosis.
Results: Two hundred ninety-nine patients participated in the study. The mean duration of interview was 12.5 minutes. There is an acceptable to good level of agreement between the GP';s (GMHAT/PC) diagnoses and the psychiatrists'; (clinical) diagnoses of any mental illness, Kappa 0.58 95% C.I (0.46, 0.72). There is good level of sensitivity (81%) and specificity (92%), with GPs correctly identifying 242 out of the 250 participants diagnosed with mental illness and 27 out of 35 of those without.
Conclusions: The finding of the study suggest that GMHAT/PC Spanish version used by GPs detected mental disorders accurately and it was feasible to use GMHAT/PC (Spanish version) in Latin America settings.

Keywords: GMHAT; Mental health assessment; Primary care mental health; Psychiatric diagnosis.

---

 

Introduction

The model of providing services for mental illness through primary health care has been recognized by the health authorities of different countries as well as the WHO. In Latin America, the vast majority of people with mental health problems have their initial contact with primary care physicians. Therefore, it is important to establish systems to help doctors in identifying people with mental health problems at the earliest opportunity and provide appropriate interventions. The GMHAT/PC (Global Mental Health Assessment Tool Primary Care and General Health Setting Version) appears promising for this purpose showing adequate levels of sensitivity and specificity in different countries where it has been evaluated^1,2.

Some people may argue that every country has to create its own assessment methods or instruments taking account of cultural and language differences. Measures of psychopathological symptoms leading to a diagnosis have been especially criticised for their universal application³. They resist using other internationally accepted tools for that reason. However all cross-cultural studies using internationally accepted diagnostic methods such as SCAN, CIDI or MINI found that these diagnostic tools accurately detect mental illness in all cultures^4-6. There may be variations in prevalence of mental disorders in different countries. In a transcultural study carried out by the WHO in the primary care attenders of 14 different countries using the same diagnostic criteria by applying the CIDI⁶ found that the overall prevalence rate of mental illness meeting ICD-10 criteria was 24%. It however ranged from a high of 52.5% in Santiago de Chile to a low 7.3% in Shanghai⁶.

An international instrument taking account of cultural and language sensitivities, following a properly carried out validation in that culture, could easily be used in the respective country⁷. To create a new tool is time consuming and expensive. That has to go through the same rigorous validation process in any case. It is therefore sensible to translate, adapt and validate an existing international tool, especially in low to middle income countries. The added advantage is that using the same tools gives useful data and information for cross-cultural and international comparisons. International data based on studies using the instrument also help in establishing the robustness of its psychometric properties.

Computer-assisted semi-structured clinical interview tools such as GMHAT/PC are not validated in Colombia and Latin America so far. Most tools that are used so far have some validity and most of them are paper based rating scales or interview schedules that had been used for research studies. GMHAT/PC on the other hand is primarily developed for clinical use. It has facility to describe the problems reported by patient, as well as recording and rating symptoms using practitioner's clinical judgement based on all the information available to him. The process of clinical assessment using GMHAT/PC is therefore very close to what is ideally expected of primary care health worker. The added advantage of getting the output in a document form with descriptive details, measurement of symptom groups and all diagnostic possibilities makes it some way ahead of all other tools available so far. It is therefore useful to translate adapt and validate GMHAT/PC to assist in providing ICD-10 based diagnosis in day to day practice in primary care and general health setting.

The study aims to assess the feasibility of using a computer assisted diagnostic interview by GPs and to examine the level of agreement between the Spanish version GMHAT/PC diagnosis and psychiatrists' ICD-10 based clinical diagnosis.

 

Methods

First, it was necessary to establish a proper and clinically relevant Spanish translation of the GMHAT/PC. The English version was translated into Spanish by a clinician who was fluent in both languages. The translated version was then reviewed in a meeting with a member of the GMHAT/PC team, the principal investigator and an experienced Spanish psychiatrist working in England. In that meetings the GMHAT/PC Spanish questions were constructed in the way that they were easily understood by patients both in Latin America Spanish speaking countries as well as in Spain. This was important to overcome country specific differences as we found that some questions needed major revision. It was necessary to retain and adjust some phrases so that Spanish GMHAT/PC can easily be used in Colombia as well as in Spain. The team compared the back translation (from Spanish to English) with the original GMHAT/PC questions and made some further necessary alterations to make the final version.

The validation study included participants ranging from those who were in remission to having symptoms of severe mental illness. They were recruited from inpatient and outpatient mental health settings. Those in the mental health setting were expected to have a wide range of psychiatric diagnoses (anxiety disorders, depression, psychosis, bipolar affective disorder, organic mental disorders, and other diagnosis). The study aimed to have approximately 50 patients with each of these diagnoses to form a sample of around 300 patients.

The study was carried out in Colombia. Patients were interviewed by GPs using GMHAT / PC. They had a full training prior to carrying out study interviews. All Patients (participants) were interviewed independently by psychiatrists with a good deal of clinical experience who were unaware of the GMHAT/PC diagnoses. They made ICD-10 based clinical diagnosis. The GMHAT/PC diagnosis and the psychiatrists' ICD-10 based clinical diagnosis were compared to examine the agreement.

 

GMHAT/PC

The GMHAT/PC is a computerized clinical assessment tool developed to assess and identify mental health problems in primary care.

The first screen is for patient information and administration of the program. The introductory screens facilitate inputting of descriptive information in the following fields: presenting symptoms, and relevant past, family, and personal problems.

The following screens consist of a series of questions leading to a comprehensive yet quick mental state assessment focusing sequentially on the following symptoms or problems: worries; anxiety and panic attacks; concentration; depressed mood, including suicidal risk; sleep; appetite; eating disorders; hypochondriasis; obsessions and compulsions; phobia; mania/hypomania; thought disorder; psychotic symptoms(delusions and hallucinations); disorientation; memory impairment; alcohol misuse; drug misuse; personality problems; stressors. One question at a time appears from these respective subsections. The questions proceed in clinical order along a tree-branch structure. For each of the major clinical disorders there are one or two screening questions. The interview moves on to the next subsection, if the patient does not have symptoms based on the screening items of a subsection.

At the end of the interview the screen asks to put the interviewer's details and the clinical diagnosis. The screen then proceeds to a menu showing the following items: a) rating scores and computer diagnosis; b) assessment and c) referral letter.

The main symptom groups on which the rating scores are based are anxiety, depression, concentration, eating disorder, hypochondriasis, phobias, obsessions, mania, psychosis, memory impairment and disorientation. In addition, there are sections for alcohol and other drug misuse, stressful events and personality difficulties.

The main computer diagnosis is derived using a hierarchical model and designed around ICD-10. The diagnostic program takes account of severity of symptoms (moderate to severe). It also generates alternative diagnosis based on presence of symptoms of other disorders.

Various studies have been dedicated to assess the psychometric properties of the tool showing results allow recommending its use in different populations. The GMHAT/PC has been proven to be useful to make accurate diagnosis in different settings and with different professions². GMHAT/PC-based diagnoses showed consistently good agreement with International Classification of Diseases, 10th Revision (ICD-10)-based clinical diagnoses made by psychiatrists in various studies⁸. It also shows reliability and validity among different psychiatrists using HADS scores as a comparator¹.

 

Results

Two hundred ninety-nine patients participated in the study, 162 (54.18%) males and 137 (45.81%) females in the age range of 14-78 (median 36.03, Standard Deviation 14.16). There was no significance difference between the gender groups with regards to age or time taken to complete GMHAT. The interviews were carried out at three mental health institutions; in total 55 (18.39%) were outpatients and 244 (81.6%) inpatients. All patients were interviewed independently by seven psychiatrists with a good deal of clinical experience.

The overall mean time taken to administer GMHAT was 12.5 min, Standard Deviation 9.98 (range 3-36 min). None of the patients declined their consent to participate in the study.

Inter-rater reliability

In this study, the inter-rater reliability was established by ratings of video recorded interviews. Two psychiatrists' ratings were taken as the gold standard. The prospective GMHAT/PC interviewers (GPs) rated the video-recorded interviews and their ratings were compared with that of psychiatrist (gold standard) to check the inter-rater reliability. Following training of GPs in using GMHAT/PC, their ratings of video recorded interviews was very similar to the ratings of the psychiatrist who recorded the interviews. Due to limited number of video recorded interviews, it was not feasible to carry out statistical analysis. This exercise intended to train the GPs in using GMHAT/PC in a consistent reliable way in the study.

Validity

The study takes account of all diagnosis made by psychiatrists as well as GMHAT/PC. Psychiatrists made a single diagnosis in 183 (61%) cases, multiple (two) diagnosis in 112 (37%) cases and multiple (three) diagnosis in another four cases. GMHAT/PC in almost all cases gave additional multiple diagnoses.

There is an acceptable to good level of agreement between the GP's (GMHAT/PC) diagnoses and the psychiatrists' (clinical) diagnoses of any mental illness, Kappa 0.58; 95% C.I (0.46, 0.72). There is good level of sensitivity (81%) and specificity (92%), with GPs correctly identifying 242 out of the 250 participants diagnosed with mental illness and 27 out of 35 of those without.

The concordance of psychiatrists' ICD-10 based clinical diagnoses and GMHAT/PC diagnoses is given in Table 1.

 

 

Concordance was considered as excellent if kappa was greater than 0.75; acceptable to good with values ​​between 0.4 and 0.74, and poor when it was less than 0.4.

Anxiety Disorders

The level of agreement for the diagnosis of anxiety disorders was slight: Kappa 0.14, 95% C.I (0.06, 0.22). Sensitivity was 100% with GPs correctly identifying 10 of the 10 participants diagnosed with anxiety disorders. The specificity was 71% with the GP correctly identifying 206 of the 289 participants not suffering from anxiety disorders.

Depression

The level of agreement for depression was acceptable to good (Kappa 0.53, 95% C.I. 0.41, 0.65). Sensitivity (84%) and specificity (86%) with the GPs correctly identifying 32 of the 38 participants diagnosed by the psychiatrists as suffering from depression and 225 out of 261 of those without.

Psychosis

The level of agreement for the diagnosis of psychosis was acceptable to good: Kappa 0.56, 95% C.I (0.46, 0.66). Sensitivity was 78% with GPs correctly identifying 91 of the 117 participants diagnosed with psychosis. The specificity was 79% with the interns correctly identifying 144 of the 182 participants not suffering from psychosis.

Mania

The level of agreement for bipolar affective disorder (mania) was acceptable to good (Kappa 0.60, 95% C.I. 0.49, 0.73). Sensitivity (63%) and specificity (95%) with the GPs correctly identifying 37 of the 59 participants diagnosed by the psychiatrists as suffering from bipolar affective disorder and 227 out of 240 of those without.

Organic Disorders

The level of agreement for the diagnosis of organic mental disorder was excellent: Kappa 0.87, 95% C.I (0.69, 1.00). Sensitivity was 88% with interns correctly identifying 7 of the 8 participants diagnosed with organic mental disorders. The specificity was 100% with the GPs correctly identifying 290 of the 291 participants not suffering from organic mental disorder.

The sensitivity, specificity, positive predicted value and negative predicted value of each diagnosis is given in Table 2.

 

 

Reliability for internal consistency

In order to assess validity, and, in particular, reliability, Cronbach's Alpha was calculated. Reliability was considered as excellent if alpha was greater than 0.9; good with values between 0.7 and 0.9, acceptable with values between 0.6 and 0.7, poor with values between 0.5 and 0.6, and unacceptable when it was less than 0.5.

Table 3 shows Cronbach's Alpha for every subscale (Symptoms). The internal consistency for depression, mania and disorientation was good with alpha values greater than 0.7. The lower value was for memory with poor internal consistency.

 

 

Feasibility

None of the participants declined participation in the study, and none gave any negative feedback. When asked what they thought of the interview, most expressed satisfaction that the GPs covered all aspects of their mental health using the GMHAT/PC. The GPs who interviewed patients found the GMHAT/PC user friendly and asked whether they could continue using it in their routine practice.

 

Discussion

The GMHAT/PC shows good agreement with psychiatrist diagnosis for any mental disorder. The tool has better agreements for the diagnosis of psychosis, depression, bipolar (mania) and alcohol and drug abuse. The agreement was excellent for the diagnosis of organic disorders. It's important to clarify that Kappa values are affected by the prevalence of the diagnosis⁹. That could explain the low agreement for anxiety disorders because those are the diagnosis with the lowest number of cases¹⁰.

The results shows that GMHAT/PC have the capacity to correctly identified patients with any mental disorder, specifically psychosis, depression and anxiety. In contrast, the sensibility for the diagnosis of mania, learning disorders, personality problems and alcohol and drug disorders is low. That's not meant necessarily a problem with the GMHAT/PC. In clinical practice is well known that learning disability, personality disorders and substance abuse are difficult diagnosis that they are not usually made in one session. Eventually, the psychiatrists need to make additional assessments and tests to be sure of the diagnosis. For this reason, is not expected that those diagnosis are easily made in a primary care level.

The GMHAT shows a very good capacity to exclude any diagnosis correctly. This applies also with the diagnosis with low sensitivity. High levels of specificity are important because demonstrate that the GMHAT/PC is more a diagnostic instrument than a screening instrument. Mental Health Gap Action Plan, developed by WHO, recommend as key actions the development of simplified diagnostic and treatment tools¹⁰. In primary care caution should be used when using screening instruments; scales measuring symptomatology tend to have low specificity leading to greater false positives. Therefore, screening instruments should be used to alert GPs that further clinical evaluation is necessary, but not to determine diagnoses. In the other side, if GPs could have diagnostic tools, the evidence suggests that with appropriate training and supervision the primary care professionals can deliver a good deal of care¹⁰.

The proportions of positive diagnosis in the GMHAT/PC that are true positive diagnosis are high for any mental illness, especially psychosis, organic disorders and alcohol and drug abuse.

The negative predicted value was very high for all diagnosis, confirming GMHAT/PC diagnosis properties.

The GMHAT/PC identified fewer patients with alcohol and drug problems than the psychiatrists. All patients were asked about substance use, that means some patients deny its use and the GP couldn't register the information in the tool.

GPs identify more cases of psychosis than the psychiatrists. Some of these patients have behavioral disorders associated to drug abuse. It could be challenging for the GP to differentiate between dysfunctional and impulsive attitudes and psychosis. The same happen with patients with borderline personality, antisocial behavior and impulse control disorders. GPs also diagnose psychosis in patients who really have maniac symptoms instead.

Some patients with psychiatrists' diagnosis of depression were identified by the GMHAT/PC as anxiety. That's not properly a mistake. In primary care the most common co-occurrence is depression and anxiety¹¹. The results of this study also demonstrate that depression score showed a significant correlation with anxiety scores. That could imply that most of the patients with depression diagnosis really have a mix anxiety-depression disorder.

GMHAT/PC identified depression in patients with psychiatrist's diagnosis of personality disorder, specifically borderline. It could be estimated that 20% to 50% of inpatients and 50% to 85% of out-patients with a current major depressive disorder have an associated personality disorder¹². Cluster B personality disorders, in particular borderline (10-30%), seem to be overrepresented¹². The coexistence of personality disorder and major depression is frequent, and most of the patients of this study could have both.

The GMHAT/PC identifies depression in patients with psychiatrists' diagnosis of alcohol abuse. Depressive symptoms are wide spread in alcohol abusing patients of all ages and are much more common than diagnosable depressive disorder¹³. The prevalence of major depression in alcohol dependence is 17%) and there is no significant correlation between severity of alcohol dependence¹³.

Some researchers have indicated that mental disorders exist across all cultures and nations while recognizing that cultural differences exist in symptom presentation and prevalence estimates¹⁴. Others have suggested the possibility that assessment tools like CIDI and other screening instruments and methods do not capture fully accurate endorsement of the disorders studied because of language or cultural differences in the conceptualization of various symptoms¹⁵. Differences in meaning of worded prompts or biases towards diagnosis of other psychopathology resulted in a decreased validity and reliability of measures that had been previously validated in English samples¹⁵. This point out the importance of examining mental health assessment with Spanish validated tools. Is important to remark the fact that Spanish is the lenguage from many different countries and regions and there are also idiosyncratic differences in language between these groups. These types of idiosyncratic differences in language can impact assessment results when the test items include words that vary by country or region¹⁶. For that reasons, in this study the GMHAT/PC was translated having several Spanish speaking professionals from different countries.

Spanish is one of the most widely spoken languages in the world; it is the official language of 21 countries and the second language in the United States. Therefore, it is relevant to have a valid Spanish version of the most used clinical interviews and assessment tools¹⁶. It's important to notice that an accurate translation does not guarantee equivalence between the original and the cross-culturally adapted versions, nor does it assure the instrument reliability or validity¹⁶. That's why this study is relevant because include the translation of the GMHAT/PC but also the reliability, feasibility and validity of the Spanish version.

The results of this study confirm its value in clinical uses of this tool. As mentioned in the previous sections, the advantage of GMHT/PC over other instruments used in primary care is its ability to provide clinically relevant diagnosis. This allows the physician to take immediate action to solve the patient's problem as far as possible without having to refer the patient to a specialist. In countries like Colombia, the waiting time to see a specialist is between one or three months. Using information technology will let GMHAT/PC based assessments to take place more easily in community settings thus improving patient experience. This could also reduce travel for patients who lives in rural areas and don't have much incomes to cover travel costs.

The GMHAT/PC could demonstrate that frontline workers can be incorporated successfully into an adequate case-detection system that is community and population based and that workers can be integrated meaningfully into the pathway to care of patients living in low resourced settings. An epidemiological study of psychiatric disorders in rural population shows that GMHAT/PC can be used as a standardized diagnosing tool in primary health care centers helping the primary care workers to diagnose psychiatric cases in a short span of time and also in referring them to specialty centers (personal communication).

One relevant use of a computer assisted tool like the GMHAT/PC may be to implement a telepsychiatry model, such as those implemented in countries where limited health care infrastructure exists¹⁷. Applying this to remote settings with no psychiatrists may contribute to prudent gains in reducing stigma by providing treatment at the community level, particularly in rural areas.

The results of this study will therefore be useful to plan and improve clinical care of people with mental disorders in primary care settings as well as to plan public health interventions.

Strengths of the Study

A good sample size and subjects is the main strength of this study. Previous GMHAT/PC studies used samples of 50 to 215 patients; this study interviewed 299 patients. This study used consultant psychiatrists' diagnoses as a gold standard rather than other measurement or diagnostic tools, trying to keep the GMHAT/PC assessment as close to routine clinical practice as possible. Consultant psychiatrists and professionals doing the assessment in the study were blind to each other's diagnoses. The GP's had no knowledge of the patients before the assessment.

Another strength of this study is a sample with varying degrees of psychopathology in different health care settings. Sample was collected in three hospitals and outpatient facilities.

Clinical diagnoses were made by trained psychiatrists in real settings based on an independent clinical interview using ICD-10 criteria. That implies that the patients included in this study are the same patients who GPs have to evaluate in their daily practice. The inclusion of these patients where many of them had more than one diagnosis made the study more robust. This not only enriched the analysis but also reflected routine clinical practice.

Limitations of the study

The relatively small number of subjects with some disorders (hypochondriasis, eating disorders) in this study could limit the analysis for this group of disorders.

The vast majority of interviews were carried out in mental health units. That implies a greater number of diagnoses of psychosis and bipolar and less cases of anxiety and depression. It's important to use the GMHAT/PC Spanish version in primary care level where the prevalence of cases is different than in a hospital setting.

 

References

1. Sharma VK, Jagawat S, Midha A, Jain A, Tambi A, Mangwani LK, et al. The Global Mental Health Assessment Tool-validation in Hindi: A validity and feasibility study. Indian journal of psychiatry. 2010; 52(4): 316-9.         [ Links ]

2. Sharma VK CJ, Krishna M, Lepping P, Bowen M. Use of the Global Mental Health Assessment Tool (GMHAT/PC) in general health care setting. 2013; In press.         [ Links ]

3. Bhui K, Mohamud S, Warfa N, Craig TJ, Stansfeld SA. Cultural adaptation of mental health measures: improving the quality of clinical practice and research. The British journal of psychiatry: the journal of mental science. 2003; 183: 184-6.         [ Links ]

4. Pull CB, Saunders JB, Mavreas V, Cottler LB, Grant BF, Hasin DS, et al. Concordance between ICD-10 alcohol and drug use disorder criteria and diagnoses as measured by the AUDADIS-ADR, CIDI and SCAN: results of a cross-national study. Drug and alcohol dependence. 1997; 47(3): 207-16.         [ Links ]

5. Kohn R, Levav I, Caldas de Almeida JM, Vicente B, Andrade L, Caraveo-Anduaga JJ, et al. Los trastornos mentales en América Latina y el Caribe: asunto prioritario para la salud pública. Rev Panam Salud Publica. 2005; 18(4/5): 229-40.         [ Links ]

6. Dy G, Lecrubier Y. Form and frequency of mental disorders across centres. Mental illness in general health care: An international study. 1995.         [ Links ]

7. Sanchez R, Echeverry J. Validating scales used for measuring factors in medicine. Rev Salud Publica (Bogota). 2004; 6(3): 302-18.         [ Links ]

8. Sharma BB, Singh S, Sharma VK, Choudhary M, Singh V, Lane S, et al. Psychiatric morbidity in chronic respiratory disorders in an Indian service using GMHAT/PC. General hospital psychiatry. 2013; 35(1): 39-44.         [ Links ]

9. Viera AJ, Garrett JM. Understanding interobserver agreement: the kappa statistic. Fam Med. 2005; 37(5): 360-3.         [ Links ]

10. WHO. mhGAP: Mental Health Gap Action Programme: scaling up care for mental, neurological and substance use disorders. Geneva: World Health Organization. 2008.         [ Links ]

11. Sartorius N, Ustün TB, Lecrubier Y, Wittchen HU. Depression comorbid with anxiety: results from the WHO study on psychological disorders in primary health care. Br J Psychiatry Suppl. 1996(30): 38-43.         [ Links ]

12. Corruble E, Ginestet D, Guelfi JD. Comorbidity of personality disorders and unipolar major depression: a review. J Affect Disord. 1996; 37(2-3): 157-70.         [ Links ]

13. Khalid A, Kunwar AR, Rajbhandari KC, Sharma VD, Regmi SK. A study of prevalence and comorbidity of depression in alcohol dependence. Indian J Psychiatry. 2000; 42(4): 434-8.         [ Links ]

14. Ballenger JC, Davidson JR, Lecrubier Y, Nutt DJ, Kirmayer LJ, Lépine JP, et al. Consensus statement on transcultural issues in depression and anxiety from the International Consensus Group on Depression and Anxiety. J Clin Psychiatry. 2001; 62 Suppl 13: 47-55.         [ Links ]

15. Asnaani A, Richey JA, Dimaite R, Hinton DE, Hofmann SG. A cross-ethnic comparison of lifetime prevalence rates of anxiety disorders. J Nerv Ment Dis. 2010; 198(8): 551-5.         [ Links ]

16. Benuto LT. Guide to psychological assessment with Hispanics. New York: Springer; 2013. xv, 422 pp.         [ Links ]

17. Thara R, Sujit J. Mobile telepsychiatry in India. World Psychiatry. 2013; 12(1): 84.         [ Links ]

 

 

Correspondence:
Paola Tejada
Psychiatrist, University of Chester
calle 127 A # 13-20 apt 201
Bogotá, Colombia
Phone 310 221 6085
E-mail: p.tejada@chester.ac.uk

Received: 19 January 2016
Revised: 14 May 2016
Accepted: 12 July 2016