Introduction

Early detection of psychological problems is, doubtless, a goal of enormous interest for diagnosis and early intervention. But it is still more important in adolescent populations, as early intervention can slow down the stabilization and progress of more severe psychopathology in later stages of life. This need has encouraged the use of screening instruments easily administered to large samples and with sufficient sensitivity and specificity to make early detection and specialized intervention possible (^{Sheldrick et al., 2015}).

One of the most widely used tests in the world is the 12-item version of the General Health Questionnaire (GHQ-12), originally proposed by ^{Goldberg (1972}). The 12-item questionnaire asks about such matters as the ability to concentrate, sleeping problems, decision-making problems, perceived stress, self-confidence or perceived happiness. It does not enable the underlying psychopathology to be established but explores perceptions of stress that could be manifestations of an underlying disorder. There are studies backing both its internal and external validity for detecting nonpsychotic psychopathology (^{Sánchez-López, & Dresch, 2008}).

Although there is wide agreement about the ability of the test to detect such signs of distress, there is persistent controversy concerning what the test really measures. The main problem is that there is no agreement on the number of dimensions that configure the test, which is an indispensable requirement for interpreting the results of its application and comparing the results found. Among the studies published are those which propose that the GHQ-12 has a one, two or three-dimensional structure.

Confusion about the structure of GHQ-12 inevitably leads to misuse and the existence of works that interpret the results in very different ways. Thus, some studies explore the "prevalence" of specific mental health problems in adolescents, such as anxiety or depression, when GHQ-12 is merely a screening test that is unlikely to be able to measure specific disorders and in any way estimate prevalence (^{Bansal, Goyal, & Srivastava, 2009}; ^{Mann et al, 2011}) or find links between such supposed disorders and other mental health problems (^{Almeida et al., 2019}; ^{Ogawa et al., 2019}; ^{Ojio, Nishida, Shimodera, Togo, & Sasaki, 2016}). Therefore, it is necessary to know the true structure of the questionnaire, the dimensions it is able to measure and, ultimately, what it really measures.

When the test has been applied in adult populations, several possible structural formulas have been found (^{Shevlin, & Adamson, 2005}), and in general, acceptable psychometric properties and diagnostic validity (^{Makowska et al., 2002}). When the test has been used in samples of adolescents, two and three-factor solutions have predominated, as summarized in Table 1. Through the statistical methods used, differences have been found in very diverse ethnic groups (^{Bowe, 2017}). It may be observed that there are two studies in Spanish adolescents, one of which found a two-factor solution, and the other, a three-factor structure.

Table 1.  Factorial studies of GHQ-12 in adolescents. 

One main reason justifying these different solutions has been suggested. There is empirical evidence that people do not answer positive responses (in the direction of distress) the same way as negative, generating two artefactual factors, the first of which is the more consistent of the two, because negative items are ambiguous about the nonexistence of such symptoms (^{Hankins, 2008}a). The methods generally used for structural analysis are inadequate for the nature of the data explored. The use of the Kaiser criterion (Eigenvalue > 1) or Scree plots are inadequate to the extent that they assume that the Likert scales represent continuum scores and tend to overestimate the presence of artefactual dimensions (^{Ruscio, & Roche, 2012}). The same is true of methods such as extraction of principal components, usually with the Pearson’s correlation matrix. These methods contribute to grouping responses based on the distribution of the variables instead of by their content. Thus, the responses to questions leading to greater ambiguity may simply be grouped by the type of response, generating some spurious factors. The Cronbach’s alpha coefficient is not an adequate estimator for studying the internal consistency of ordinal scales either, nor are Likert responses (^{Crutzen, & Peters, 2017}; ^{Yang, & Green, 2011}). Several studies have shown that when these biases are controlled for and appropriate statistics are applied, the solution found is always one-factor (^{Hankins, 2008b}; ^{Rey et al., 2014}; ^{Romppel et al., 2013}).

Another controversial question is whether the interpretation of the results of the GHQ-12 in adolescents should respond to the same rules as for adults, that is, place the cutoff point at three or more affirmative answers. There are no studies exploring the predictive validity of the GHQ-12 in Spanish adolescent populations. When it has been applied to samples from other countries, the data are very contradictory. Some suggest the scores of adolescents should be interpreted the same way as adults (^{French, & Tait, 2004}), others use Likert scores, finding high prevalence of psychopathological problems (^{Bansal, Goyal, & Srivastava, 2009}), or propose cutoff points of 10 for males and 11 for females on continuum scores (^{Baksheev et al., 2011}). The author of the GHQ-12 suggested using the sample mean as the cutoff point when no previous data are available (^{Goldberg, & Williams, 1998}).

The purpose of this study was to explore the dimensionality of the GHQ-12 in a representative sample of adolescents enrolled in schools in a large Western city (Madrid, Spain), using the most appropriate methods, able to control for the biases mentioned. In addition, the differences between males and females in the sample were estimated according to the factor solution found.

Method

Results

The outlier detection study did not advise excluding any of the subjects from the simple (Mahalanobis Distance p>.001 in all cases).

Likert-type scoring

The descriptive statistics for the items are shown in Table 2.

Table 2.  Descriptive statistics of the GHQ-12 items, Likert form. 

The Mardia test was applied to the Pearson correlations matrix, which showed that the distribution of items did not meet multivariate normality (p<.05), so a polychoric correlations matrix was constructed. Based on the new matrix, an optimized parallel analysis was made which suggested one-dimensionality of the scale with a single factor explaining 46% of accumulated variance (Table 3). The goodness-of-fit statistics for this one-factor solution were satisfactory: RMSEA=.07; NNFI=.97; CFI=.98; GFI=.98; AGFI=.98. The model repeatability index (Latent h =.91; Observed h =.81) was adequate, as was the suitability of the one-factor solution (UniCo = 0.97; ECV= 0.87). Internal consistency indicators were also appropriate: GLB=.93; ω=.89; α_s=.89. Table 4 shows the characteristics of the one-factor solution found. The results did not vary when the same tests were done separately in males and females.

Table 3.  One-factor solution (Likert scoring). 

Criterion Scoring

The descriptive statistics of the items are shown in Table 4. The Mardia test showed again that the distribution of the items did not meet multivariate normality (p<.05), so the tetrachoric correlations matrix was constructed. An optimized parallel analysis based on the new matrix suggested the unidimensionality of the scale with a single factor explaining 60.2% of the total variance of the test (Table 5). The goodness-of-fit statistics for this one-factor solution were satisfactory: RMSEA=.04; NNFI=.99; CFI=.99; GFI=.99; AGFI=.99. The model’s Repeatability Index (Latent-h =.95; Observed-h =.70) and the suitability of the one-factor solution (UniCo=.99; ECV=.92) were also adequate. The internal consistency indicators were appropriate as well: GLB=.96; ω=.94; α_s=.94. Table 5 shows the characteristics of the one-factor solution found. The results did not vary when the same tests were given males and females separately.

Table 4.  Descriptive statistics for the GHQ-12 items, criterion form. 

Table 5.  One-factor solution (criterion scoring). 

Sex differences

Table 6 shows the differences in answers to the GHQ-12 between males and females. Females said they had more problems concentrating, more depressive symptoms, less self-confidence and less perceived ability to cope with problems, but especially, more feelings of anxiety and tension than males, although all of this with a very small effect size.

Table 6.  Gender differences in the responses to GHQ-12 items (Likert scoring). 

Scores

Following the author’s instructions, the cutoff point should be three affirmative answers as an indicator of risk of poor mental health. This risk would be moderate for the mean plus the standard deviation (three, four or five affirmative answers) and severe for over six affirmative answers. Thus, the sample could be classified in three categories: no risk, moderate risk and severe risk (Table 7).

Table 7.  Distribution of the sample classified into three categories of risk of poor mental health. 

Another way of classifying the participants in our sample would be to use the Likert scores, with which scores not over the sample mean plus the standard deviation show no risk of poor mental health, between the mean plus one and plus two standard deviations show moderate risk, and over the mean plus two standard deviations show severe risk (Table 7).

Discussion

The results of this study clearly show that the GHQ-12 should be considered a unidimensional test of psychological distress, at least when applied to an adolescent population. Exploratory and semi-confirmatory (^{Lorenzo-Seva, & Ferrando, 2013}) factor analysis were used, controlling for biases inherent in the item response method, in a randomized sample representative of the adolescent population (14-16-years-old) in a large city (Madrid). Unlike previous studies, usually using convenience sampling, obsolete methods inadequate for the nature of the data (Kaiser criterion, Scree-plot, Pearson correlation matrix, Cronbach’s alpha, etc.), were avoided and more appropriate statistical indicators were used, both for the Likert-type response format and dichotomous scoring of the questionnaire. All of the statistics enable us to state that there is no doubt at all of the one-dimensionality of the test. These data are consistent with the proposals of the authors who attribute the appearance of two or three dimensions to the use of inadequate estimators, which are based on inadequate assumptions, such that when all those biases are controlled for the solution is always unidimensional (^{Hankins, 2008}).

As a unidimensional test, the GHQ-12 is a reliable instrument, both overall all and each item alone. What in other studies appear to be independent dimensions (social dysfunction, anxiety and loss of self-confidence) are simply components of a general construct of psychological distress, from which diagnostic approaches cannot be derived, but merely initial screening, which must be followed up with specific psychodiagnosis.

When gender differences were studied, females scored higher on many of the symptoms and the overall test score, as is usual. The real reasons for this tendency of females to report more distress, somatic and psychic symptoms is unknown (^{Davis, Matthews, & Twamley, 1999}), a good number of possible reasons having been proposed: innate differences in somatic and visceral perception, differences in labelling, the description and reporting of symptoms, differences in education and socialization, sex differences in the incidence of abuse and violence, and gender prejudices in research as well as in clinical practice (^{Barsky, Peekna, & Borus, 2001}). Whatever the reason, the data in our study suggest that adolescent girls are more vulnerable than their male peers.

Applying the usual criterion scoring, we found that 30% of male and 42% of female adolescents were at risk of developing mental health problems. Doubtless, one of the main characteristics of this period of life is emotional instability derived from the change in roles, search for identity, the beginning of autonomy in decision-making, etc., which coincides with a neurologically critical period of synaptic pruning (^{Blakemore, & Choudhury, 2006}). However, it seems exaggerated for the percentages mentioned to have any kind of clinical entity. In any case, a screening instrument like the GHQ-12 can enable the early detection of problems which often form the basis for establishment of psychopathological symptoms that persist through adulthood (^{Paus, Keshavan, & Giedd. 2008}). The results found using the Likert scores (around 11% of males and 18% of females at risk of poor mental health) are closer to clinical reality, but it should not be forgotten that the GHQ-12 is a screening instrument and not meant for diagnosis.

In conclusion, the GHQ-12 is a unidimensional screening test for psychological distress, with excellent psychometric properties for its application in an adolescent population. Future studies should replicate these results in other populations (e.g., rural, adults, elderly, etc.), in other cultural settings and in clinical populations. The use of adequate statistical methods can enable old controversies to be overcome and favor proper application and interpretation of the results provided by the test.