SciELO - Scientific Electronic Library Online

 
vol.38 issue1Losing and regaining organizational attractiveness during the recruitment process: a multiple-degment factorial vignette study author indexsubject indexarticles search
Home Pagealphabetic serial listing  

My SciELO

Services on Demand

Journal

Article

Indicators

Related links

  • On index processCited by Google
  • Have no similar articlesSimilars in SciELO
  • On index processSimilars in Google

Share


Revista de Psicología del Trabajo y de las Organizaciones

On-line version ISSN 2174-0534Print version ISSN 1576-5962

Rev. psicol. trab. organ. vol.38 n.1 Madrid Apr. 2022  Epub Apr 04, 2022

https://dx.doi.org/10.5093/jwop2022a5 

ARTICLES

Occupational depression in a Spanish-speaking sample: associations with cognitive performance and work-life characteristics

Depresión ocupacional en una muestra hispanohablante: asociaciones con el rendimiento cognitivo y características de la vida laboral

Renzo Bianchia  , Guadalupe Manzano-Garcíab  , Pilar Montañés-Murob  , Emily A Schonfeldc  , Irvin S Schonfeldd 

aInstitute of Work and Organizational Psychology, University of Neuchâtel, Neuchâtel, NE, Switzerland;

bUniversity of La Rioja, Logroño, Spain;

cWeill Cornell Medical Center, New York City, NY, USA;

dThe City College of the City University of New York, New York City, NY, USA

ABSTRACT

This 386-participant study investigated the structural and psychometric properties of the Spanish version of the Occupational Depression Inventory (ODI). Exploratory structural equation modeling bifactor analysis revealed that the ODI meets the requirements for essential unidimensionality. Measurement invariance held across our sample and the English- and French-speaking samples used in the ODI's initial validation study. Mokken scale analysis indicated that (a) the scalability of the instrument was strong, (b) no violations of monotonicity or local independence were present, and (c) invariant item ordering was sufficiently accurate. The ODI's reliability was optimal. The ODI exhibited both convergent validity and discriminant validity vis-à-vis a job-unrelated measure of depression. Furthermore, occupational depression correlated substantially, and in the expected direction, with objective cognitive performance and 10 widely studied work-life characteristics. This study suggests that the ODI's Spanish version has excellent structural and psychometric properties and can be confidently employed by occupational health specialists.

Keywords Bifactor analysis; Cognition; Cross-cultural research; Expected value of control; Mokken scale analysis; Occupational Depression Inventory; Work stress

RESUMEN

El presente estudio, realizado con 386 participantes, ha indagado en las propiedades estructurales y psicométricas de la versión española del Inventario de Depresión Ocupacional (ODI, según sus siglas en inglés). Un análisis bifactorial de un modelo exploratorio de ecuaciones estructurales mostró que el ODI reúne los requisitos de unidimensionalidad esencial. La invarianza de la medida se mantuvo en nuestra muestra y en las muestras anglo y francoparlantes utilizadas en el estudio de validación inicial. Un análisis Mokken de la escala indica que: a) el instrumento tenía un fuerte carácter escalar, b) no hubo violación de la monotonicidad o independencia local y c) el orden invariante de los ítems fue lo suficientemente preciso. La fiabilidad ha sido óptima. El ODI presentó tanto validez convergente como discriminante en relación con una medida de depresión no relacionada con el trabajo. Además, la depresión laboral correlacionó en gran medida, en la dirección esperada, con el desempeño cognitivo objetivo y 10 características muy estudiadas de la vida laboral. El estudio destaca que la versión española del ODI tiene unas propiedades estructurales y psicométricas excelentes, por lo que puede utilizarse con toda confianza por los especialistas en salud ocupacional.

Palabras clave Análisis bifactorial; Cognición; Investigación transcultural; Valor de control esperado; Análisis de la escala de Mokken; Inventario de Depresión; Ocupacional; Estrés laboral

Introduction

The Occupational Depression Inventory (ODI) was recently created to assess work-attributed depressive symptoms and identify likely cases of job-ascribed depression (Bianchi & Schonfeld, 2020, 2021b; Schonfeld & Bianchi, 2021). The ODI was developed in a context in which (a) job-related distress and its potentially lethal consequences have become a focal concern among occupational health specialists (Gonzalez-Mulé & Cockburn, 2021; Hassard et al., 2018; Howard et al., 2021; Matthews et al., 2021; Melchior et al., 2007; Tiesman et al., 2015) and (b) substantial definitional and measurement problems have been found to affect the popular construct of “burnout” (Bianchi & Laurent, 2018; Bianchi & Schonfeld, 2021c; Bianchi et al., 2020; Bianchi, Verkuilen et al., 2021; Cox et al., 2005; Meier & Kim, 2021; Mirkovic & Bianchi, 2019; Rotenstein et al., 2018; Schwenk & Gold, 2018; Vinkers & Schaafsma, 2021).1 The burnout construct is notably undermined by (a) its lack of validity (e.g., its lack of discriminant validity vis-à-vis the depression construct; Ahola et al., 2014; Bianchi et al., 2019; Bianchi, Verkuilen et al., 2021), (b) its shaky clinical and theoretical foundations (Schaufeli, 2003; Schwenk & Gold, 2018), (c) its neglect of key aspects of job-related distress (e.g., work-related suicidal thoughts; Schaufeli & Enzmann, 1998), and (d) the fact that the burnout syndrome is undiagnosable and nosologically uncharacterized—with problematic consequences for case identification, treatment development, prevalence estimation, public health policymaking, or worker eligibility for sick pay (Bianchi, 2017; Bianchi & Schonfeld, 2021c; Rotenstein et al., 2018; Schwenk & Gold, 2018).

Because depressive symptoms constitute a basic human response to insurmountable adversity (or unresolvable stress), even among individuals with no noticeable susceptibility to clinical depression (Bianchi et al., 2020; Dohrenwend, 2000; Dura et al., 1990; Gilbert, 2006; Pryce et al., 2011; Wichers, 2014; Willner et al., 2013), addressing the issue of job-related distress in the area of depression research appears to be particularly relevant (Bianchi, Verkuilen et al., 2021; Schwenk & Gold, 2018). The ODI was designed with reference to the diagnostic criteria for major depression found in the latest edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5; American Psychiatric Association, 2013).2 The ODI thus assesses anhedonia, depressed mood, sleep alterations, fatigue/loss of energy, appetite alterations, feelings of worthlessness, cognitive impairment, psychomotor alterations, and suicidal ideation within a two-week time window. The ODI assesses depressive symptoms that individuals attribute to their jobs, in contrast to assessing depressive symptoms in the “cause-neutral” manner that is typical of other depression scales (Bianchi & Schonfeld, 2020). The reliance on causal attributions has been commonplace in (occupational) health research. As an illustration, the Stress in America™ survey commissioned by the American Psychological Association has made use of causal attributions to identify leading sources of stress among the general public (American Psychological Association, 2015). Moreover, causal attributions are crucial to the diagnosis of several well-known disorders, such as posttraumatic stress disorder (PTSD). The symptoms characterizing PTSD derive their diagnostic value from being attributable to specific traumatic/stressful events (American Psychiatric Association, 2013). An idea underlying the reliance on causal attributions is that affected individuals are often in a privileged position to synthesize information on what goes wrong in their lives, especially when “low-observability” phenomena are at stake; in many cases, no one else has access to more or better information (Baldwin, 2000; Bianchi & Schonfeld, 2020; Roe et al., 2022).

The ODI allows investigators to adopt both dimensional (continuum-based) and categorical (diagnostic) approaches to workplace depression. On the one hand, the ODI assesses the severity of work-attributed depressive symptoms. On the other hand, the ODI incorporates a clinically informed algorithm for establishing provisional diagnoses of job-ascribed depression. This dual lens is in keeping with the growing coordination of dimensional and categorical approaches in psychopathological science (Bianchi, 2020; Bianchi, Verkuilen, et al., 2021; Haslam et al., 2012; Kotov et al., 2017). Dimensions and categories are regarded here as two complementary perspectives on the phenomenon of interest (Pickles & Angold, 2003). Depending on an investigator's goals, one perspective may be more relevant or implementable than the other. For instance, an investigator interested in estimating the prevalence of occupational depression, or in deciding whether an intervention should be launched, may need to adopt a primarily categorical perspective. An investigator interested in the intricate processes at stake along the continuum of occupational depression will likely choose a dimensional perspective.

The research conducted on the structural and psychometric properties of the ODI suggests that the instrument has high factorial validity and strong reliability (Bianchi & Schonfeld, 2020, 2021a, 2022; Hill et al., 2021). Relying on exploratory structural equation modeling (ESEM) bifactor analysis, Bianchi and Schonfeld (2020) approached the structure of the ODI through a general Occupational Depression factor and two specific factors—on account of the anhedonic-somatic and dysphoric symptom items of the scale. The ODI demonstrated “essential unidimensionality” (Rodriguez et al., 2016), with the general factor explaining about 89% of the common variance extracted. Cronbach's alphas and McDonald's omegas for the ODI are around .90. From the standpoint of criterion validity, evidence indicates that the ODI correlates, in the expected direction, with various work and nonwork factors, including job satisfaction, dedication to work, willingness to stay in the job, social support in work life, active search for another job or position, trait anxiety, general health status, life satisfaction, and objective cognitive performance (Bianchi & Schonfeld, 2020, 2021a, 2022; Hill et al., 2021). The ODI has been examined in France, New Zealand, South Africa, and the USA to date. The instrument has been validated in English and French.

The main purpose of the present study was to investigate the structural and psychometric properties of the Spanish version of the ODI. We examined the instrument's properties using advanced statistical techniques anchored in both classical test theory and item response theory. We relied on ESEM bifactor analysis (Marsh et al., 2014; Rodriguez et al., 2016) and Mokken scale analysis (Meijer & Baneke, 2004; Mokken, 1971; Stochl et al., 2012). In addition to scrutinizing “intrinsic” properties of the ODI (e.g., factorial structure and essential unidimensionality, scalability, monotonicity, local independence, invariant item ordering, test-score reliability), we examined the behavior of the ODI as it relates to a cause-neutral measure of depressive symptoms. Based on the view that, at a population level, all individuals with occupational depression should be identified as depressed in a cause-neutral assessment of depression whereas only some of the individuals identified as depressed in a cause-neutral assessment of depression should ascribe their depressive symptoms to their job, we expected the ODI to show both a degree of convergent validity and a degree of discriminant validity vis-à-vis a cause-neutral measure of depressive symptoms.

For the purpose of evaluating the criterion validity of the ODI, we inquired into the association of the ODI with objective cognitive performance. In light of (a) the state of the art on the links between depression and neuropsychological functioning (e.g., Rock et al., 2014; Snyder, 2013) and (b) two recent ODI studies conducted in France (Bianchi & Schonfeld, 2021a, 2022), we hypothesized that the ODI would predict cognitive performance negatively. Finally, we investigated the links between occupational depression and widely studied work-life characteristics, namely, interpersonal conflict at work, job incivility, unreasonable work tasks, unnecessary work tasks, work overload, social support at work, job autonomy, skill development, job recognition, and job meaningfulness (Bianchi et al., 2021; Guthier et al., 2020; Matthews et al., 2021; Melchior et al., 2007; Niedhammer et al., 2015; Schaufeli et al., 2009; Semmer et al., 2015; Sibeoni et al., 2021). We hypothesized that the ODI would show positive associations with the former five variables and negative associations with the latter five variables.

Spanish is a widely spoken language, with about 500 million native speakers worldwide—mainly in Spain and the Americas (SIL International, 2021). Rendering the ODI available in Spanish is thus an important step in expanding the accessibility of the instrument. At a more general level, the present study offers an opportunity to learn more about the characteristics of the ODI. The need for (more) rigorous examinations of psychological scales has been repeatedly emphasized, notably in applied psychology (see Cortina et al., 2020; see also Hussey & Hughes, 2020). By employing both ESEM bifactor analysis and Mokken scale analysis, this study submits the ODI to a particularly detailed and strict examination.

Method

Study Sample

The present study involved 386 Spanish working individuals (71% female) from various occupational groups (e.g., educational staff members, health professionals, social workers). The year-rounded mean age in the study sample was 46 (SD = 9, range = 24-65). A vast majority of the participants (about 86%) were employed full-time. Participants were recruited through advertisements in social media and email contacts with organizations and professional associations. Recruitment took place in May and June 2021. Participation in the study was voluntary and without compensation. Full confidentiality was guaranteed to all participants. Consent to participate was requested. The study complied with the ethical standards of the institutional review board of the University of Neuchâtel. The study was conducted online using Qualtrics (https://www.qualtrics.com/).

Measures of Interest

ODI. The ODI was designed with reference to the DSM-5's diagnostic criteria for major depression (American Psychiatric Association, 2013). The instrument comprises nine core symptoms items (“My experience at work made me feel like a failure”), rated from 0 (never or almost never) to 3 (nearly every day), and a subsidiary question related to turnover intention, associated with three response options (Yes, No, and I don't know). Participants are surveyed about how they felt over the past two weeks. The ODI includes detailed instructions to respondents that play an important role in the administration of the scale. Respondents are invited to consider various sources for their symptoms, including work-unrelated and unknown sources. This precaution is intended to discourage hasty symptom attributions to work. Respondents are asked to select the zero frequency if they believe a symptom originates from a nonwork source or if the source of the symptom is unknown to them. Using the ODI, the investigator can (a) quantify depressive symptoms that respondents specifically attribute to their work and (b) ascertain whether respondents qualify as likely cases of job-ascribed depression using a dedicated diagnostic algorithm—the diagnostic algorithm is detailed in Bianchi and Schonfeld's (2020) article; the paper also contains SPSS syntax for the diagnostic algorithm.

We translated the ODI into Spanish using a back-translation method (Streiner et al., 2015). First, the English version of the instrument was translated into Spanish by a native Spanish speaker fluent in English. Second, the Spanish version of the instrument was translated back into English by a native English speaker fluent in Spanish. Neither the English-to-Spanish nor the Spanish-to-English translators had prior familiarity with the measure. The two translators did not know each other. Third, the English version resulting from the back-translation and the original English version were carefully compared. No problematic discrepancies were identified. The items of the ODI translated into Spanish are displayed in Table 1, together with their English counterparts.

Table 1.  Spanish Version of the Items of the Occupational Depression Inventory (ODI) 

Note. The full ODI form (that includes the instructions to respondents) is available in Spanish in Supplementary Material 1, together with an SPSS syntax implementing the provisional diagnosis algorithm of the ODI. SQ = subsidiary question.

As expected with a nonclinical sample, the distribution of ODI mean scores was positively skewed (skew = 1.065, SE = 0.124). Still, ODI mean scores ranged from 0.000 (minimum possible score) to 3.000 (maximum possible score), thus covering the continuum of occupational depression. Of our 386 respondents, about 68% (n = 264) scored between 0.000 and 0.999, about 25% (n = 96) scored between 1.000 and 1.999, and about 7% (n = 26) scored between 2.000 and 3.000. About 4% of our participants (n = 16) met the criteria for a provisional diagnosis of job-ascribed depression. About 13% of our participants (n = 52) indicated that they were considering leaving their current job or position due to their distress at work. The full Spanish version of the ODI, including the instructions to respondents, is available in Supplementary Material 1.

“Cause-neutral” depressive symptoms. We assessed “cause-neutral” depressive symptoms with the Spanish version of the seven-item Depression subscale of the Depression Anxiety Stress Scales-21 (DASS-21-D; Daza et al., 2002). The DASS-21-D is a popular measure of depressive symptoms (Lee et al., 2019). The items of the DASS-21-D are rated on a 4-point scale (from 0 for did not apply to me at all to 3 for applied to me very much or most of the time). The DASS-21-D covers the past week. McDonald's omega (ω) was .903.

Objective cognitive performance. Objective cognitive performance was assessed with an extended, six-item version of the Cognitive Reflection Test (CRT; see Primi et al., 2016; see also Bianchi & Schonfeld, 2022). This extended version includes the three original CRT items created by Frederick (2005) as well as three additional items. The six items are available in Primi et al.'s (2016) article. The CRT consists of problems that require, in order to be successfully resolved, the inhibition of a heuristic, intuitive response and an engagement in more effortful reasoning. The CRT thus mobilizes working memory resources and helps characterize decision-making styles by identifying more or less controlled (or more or less automatic) forms of information processing (Frederick, 2005). There is meta-analytic evidence that the CRT predicts training proficiency (i.e., the degree of technical skill and competence acquired after a period of education or instruction) and job performance (Otero et al., 2021). Here is a sample item:

“A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much does the ball cost?”. The heuristic, intuitive answer is 10 cents; the correct answer is 5 cents.3

Participants were asked to resolve each of the six problems that the CRT comprised by selecting, among four response options, the correct response. The four response options included: (a) the correct answer, (b) an intuitive, but incorrect, answer, (c) a nonintuitive, incorrect answer, and (d) a “other” answer, which was always incorrect. Participants were given 60 seconds to resolve each problem. A nonresponse was considered incorrect. Any incorrect answer was coded 0, and a correct answer was coded 1, leading to a mean score ranging from 0 to 1 for each respondent. McDonald's ω was .826. We provide detailed descriptive statistics pertaining to the CRT in Supplementary Material 2.

Work-life characteristics. We assessed 10 work-life characteristics commonly studied in occupational health research: interpersonal conflict at work, job incivility, unreasonable work tasks, unnecessary work tasks, work overload, social support at work, job autonomy, skill development, job recognition, and job meaningfulness (Bianchi et al., 2021; Guthier et al., 2020; Matthews et al., 2021; Melchior et al., 2007; Niedhammer et al., 2015; Schaufeli et al., 2009; Semmer et al., 2015; Sibeoni et al., 2021). Each work-life characteristic was evaluated using a single item covering the past week (the items can be found in Supplementary Material 3). Participants used a 5-point rating scale to respond (from 1 for strongly disagree to 5 for strongly agree). We translated the items from English to Spanish using a back-translation procedure similar to that we used in creating the Spanish version of the ODI. By relying on single-item measures, we aimed to cover a vast array of work-life characteristics without rendering involvement in the study too lengthy and burdensome to participants (Bowling, 2005). Such a concern is worthy of consideration to encourage initial participation in a study and limit attrition among enrolled participants.

Data Analyses

We first examined the factorial structure of the ODI using ESEM bifactor analysis (Marsh et al., 2014). We relied on a partially specified target rotation (PSTR). Compared to common-practice CFA, an advantage of the PSTR is that nontarget loadings are “not” fixed to be equal to 0; instead, they are “encouraged” to get as close to 0 as possible by the loss function, allowing factorial complexity to be modeled. Based on Bianchi and Schonfeld's (2020) findings regarding the factorial structure of the ODI, we considered two specific factors (or bifactors) in addition to the general Occupational Depression factor. The first specific factor targeted the “anhedonic-somatic” items of the ODI (Items 1, 3, 4, 5, 7, and 8), and the second specific factor targeted the “dysphoric” items of the ODI (Items 2, 6, and 9). We treated the ODI items as ordinal and used the weighted least squares—mean and variance adjusted—(WLSMV) estimator. We conducted our ESEM bifactor analysis in Mplus 8 (Muthén & Muthén, 1998-2017). To estimate the extent to which the general factor accounted for the common variance extracted, we computed the explained common variance (ECV) statistic (Rodriguez et al., 2016). An ECV index exceeding .80 is suggestive of essential unidimensionality. In addition, we used the ω and ω hierarchical (ωH) coefficients to further inquire into the ODI's reliability and the general factor's correlation with the observed total scores.

We capitalized on data collected in France (n = 1,450), New Zealand (n = 492), and the U.S. (n = 312) in the context of the initial validation study of the ODI (Bianchi & Schonfeld, 2020) to investigate the measurement invariance of a unidimensional model across our Spanish sample and previously examined ODI samples. We focused on: (a) configural invariance, which concerns the equivalence of model forms; (b) metric invariance, which concerns the equivalence of factor loadings; and (c) scalar invariance, which concerns the equivalence of item thresholds (Putnick & Bornstein, 2016). As recommended with ordinal factor analysis (Shi et al., 2020, 2021), we scrutinized delta change in CFI (ΔCFI) and delta change in SRMR (ΔSRMR). We relied on conservative thresholds for flagging invariance violations; the thresholds were -.005 for ΔCFI and .005 for ΔSRMR (Putnick & Bornstein, 2016; Rutkowski & Svetina, 2017). In addition, we estimated measurement invariance across sexes within our Spanish sample, which included 273 women and 113 men.

We further examined the characteristics of the ODI based on Mokken scale analysis (Meijer & Baneke, 2004; Sijtsma & Molenaar, 2002; Stochl et al., 2012). We conducted the analysis using the Mokken package version 3.0.3 (van der Ark, 2012) in R version 4.0.3 (R Core Team, 2020). As recommended by Sijtsma and van der Ark (2017), and in accordance with the monotone homogeneity and double monotonicity models, we focused on the scalability (homogeneity), monotonicity, local independence, and invariant item ordering (IIO) properties.

The scalability property implies that the endorsement of more “difficult” items is associated with a higher probability of endorsing “easier” items (Dima, 2018). In the context of a depression scale such as the ODI, item difficulty is equivalent to the “severity” of the symptom assessed by each item (Meijer & Egberink, 2012). For instance, the ODI's suicidal ideation item is expected to be more difficult, or less commonly endorsed, than, say, the ODI's sleep alterations item because suicidal ideation denotes a more severe and pathognomonic symptom of depression compared to sleep alterations. The scalability property is examined based on the H coefficient, considered at the level of items, item pairs, and the scale (Sijtsma & van der Ark, 2017). Item-level H coefficients should be > .30. Pairwise H coefficients should be > 0. A scale is considered weak if .30 ≤ H < .40; moderate, if.40 ≤ H < 0.50; and strong, if H ≥ .50. The predicates “weak,” “moderate,” and “strong” are indicative of the extent to which the ordering of individuals by test score reflects the ordering on the latent variable.

The monotonicity property implies that the item response function is monotonically nondecreasing. In other words, as one moves up on the latent variable, the probability of endorsing an item should not decrease. Monotonicity violations were examined in terms of their presence, statistical significance, and seriousness by means of the crit statistic. A crit statistic below 80 suggests that no serious violations are at stake (e.g., van Schuur, 2003).

The local independence property implies that the latent variable explains the associations among the items. Put differently, the items are assumed to be unrelated to each other when the latent variable is controlled for. We examined the local independence property based on the conditional association procedure (Straat et al., 2016), which relies on so-called W indices flagging locally dependent item pairs (Sijtsma & van der Ark, 2017).

The IIO property implies that the order of difficulty of the items should be the same at all levels of the latent variable (Dima, 2018). IIO violations were examined based on the manifest IIO method (MIIO) and the coefficient HT (Ligtvoet et al., 2010). The MIIO method identifies IIO violations in terms of their presence, statistical significance, and seriousness. Seriousness was, again, indexed by the crit statistic, a crit statistic reaching 80 being indicative of serious violations. Assuming that MIIO holds, HT < .30 indicates that the item ordering is too inaccurate to be useful; .30 ≤ HT < .40 indicates weak accuracy; .40 ≤ HT < .50 indicates moderate accuracy; and HT ≥ .50 indicates strong accuracy. These properties are described in detail in Dima's (2018) and Sijtsma and van der Ark's (2017) articles, among many others.

We investigated the convergent validity and discriminant validity of the ODI and the DASS-21-D mainly based on ESEM bifactor analysis in Mplus 8 (Muthén & Muthén, 1998-2017). We relied on a PSTR. We treated all items as ordinal and, again, used the WLSMV estimator. We scrutinized the base associations among our variables of interest through Pearson and Spearman correlational analyses.

Results

ESEM Bifactor Analysis

The model under examination, which involved two specific factors in addition to the general factor, displayed a satisfactory fit: RMSEA = .025, CFI = 1.000, TLI = .999, WRMR = 0.267, χ2(12) = 14.991. All ODI items loaded strongly on the general factor (mean loading on the general factor = .839, SD = .058), and more strongly on the general factor than on any of the two specific factors (Figure 1). Judging from the factor loadings, the specific factors, especially the Dysphoric bifactor, were relatively weak. As per the ECV statistic, the general factor accounted for about 88% of the common variance extracted. The ECV statistic thus indicated essential unidimensionality.

Figure 1.  Exploratory Structural Equation Modeling Bifactor Analytic Model of the Occupational Depression Inventory-Factor Loadings. Target loadings are bolded. Note. OD = General Occupational Depression factor; ANH-SOM = Anhedonic-Somatic bifactor; DYS = Dysphoric bifactor. N = 386. 

With a value of .917, ωH was high. We computed the square root of ωH to obtain the correlation between the general factor and the observed total scores (Rodriguez et al., 2016). The correlation was as high as .958. Dividing ωH (.917) by ω (.972),4 we found that almost all of the reliable variance in the observed total scores (95%) was attributable to the general factor. All in all, the ODI was “unidimensional enough” for a unidimensional measurement model to be specified in an SEM context when needed.

Measurement Invariance

We first examined the measurement invariance of a unidimensional model across our sample and the three samples used in the initial validation study of the ODI (Bianchi & Schonfeld, 2020). We found measurement invariance to hold across the four samples under scrutiny, even by the most conservative standards (e.g., Rutkowski & Svetina, 2017). The CFI and SRMR barely varied as additional constraints were imposed (Table 2). ΔCFI was .000 between the configural model and the metric model, and -.001 between the metric model and the scalar model; ΔSRMR was .002 between the configural model and the metric model, and .001 between the metric model and the scalar model. Although the RMSEA and TLI were not our indicators of interest in the context of ordinal factor analysis, we note that the RMSEA never increased, and the TLI never decreased, from one model to another. Such results are consistent with the notion that measurement invariance across samples was highly satisfactory.

Table 2.  Measurement Invariance across Samples 

Note. The analysis involved four samples. The first three samples are the original samples on which the Occupational Depression Inventory was initially validated (Bianchi & Schonfeld, 2020). The fourth sample is our Spanish sample (N = 386).Note. CFI = CFI = comparative fit index; ΔCFI = delta (change in) CFI; SRMR = standardized root mean squared residual; ΔSRMR = delta (change in) SRMR; df = degrees of freedom.

Turning to measurement invariance across sexes within our Spanish sample, we found that (a) CFI never decreased as constraints were added and (b) SRMR increased by only .001 from the configural model to the metric model, and from the metric model to the scalar model. RMSEA never increased and TLI never decreased. Measurement invariance across sexes was thus strongly supported.

Table 3.  Mokken Scale Analysis of the Occupational Depression Inventory 

Note. N = 386; H: scale-level H; Hi: item-level H; SE = standard error; 95% CI = 95% confidence interval; HT = invariant item ordering; MIIO = manifest invariant item ordering method; #vi = violations; #zsig and #tsig = significant violations; crit = serious violations-items for which the crit statistic reaches 80 seriously violate requirements; ODI1 = anhedonia; ODI2 = depressed mood; ODI3 = sleep alterations; ODI4 = fatigue/loss of energy; ODI5 = appetite alterations; ODI6 = feelings of worthlessness; ODI7 = cognitive impairment; ODI8 = psychomotor alterations; ODI9 = suicidal ideation.

Mokken Scale Analysis

Results are summarized in Table 3. We found the overall scalability of the ODI to be strong, with a scale-level H value of .718 (95% CI = .673, .758) having a SE of only .022 (Sijtsma & van der Ark, 2017). Item-level H values ranged from .658 for Item 9 (95% CI = .516, .758, SE = .061) to .742 for Item 7 (95% CI = .693, .783, SE = .023). No pairwise H showed a low value. The Automated Item Selection Procedure signaled a single scale involving all ODI items up to a (very high) threshold of .65 (Stochl et al., 2012). We did not observe any violation of monotonicity. The conditional association procedure did not flag any item, meaning that there was no violation of local independence (Sijtsma & van der Ark, 2017). No statistically significant or serious violation of IIO was detected. IIO was satisfactory, with an HT of .487 (Ligtvoet et al., 2010). The most difficult ODI item was Item 9 (suicidal ideation); the easiest ODI item was Item 4 (fatigue/loss of energy).

Reliability

We examined the reliability of the ODI based on McDonald's ω,5 Guttman's lambda-2 (λ2), and the Molenaar-Sijtsma statistic (MS). McDonald's ω was .964; λ2, .938; and the MS, .940. All indices thus suggested that the ODI is a highly reliable measure.

Convergent Validity, Discriminant Validity, and Criterion Validity

Our ESEM bifactor analysis of the items of the ODI and the DASS-21-D involved two bifactors—one linked to the ODI items and the other linked to the DASS-21-D items—in addition to a general Depression factor. The model showed a satisfactory fit, RMSEA = .049, CFI = .994, TLI = .990, WRMR = .518; χ2(75) = 144.487, and supported both the convergent validity and the discriminant validity of the two measures (Supplementary Material 4). On the one hand, every item of the ODI and the DASS-21-D loaded substantially on the general factor (M =.660, SD =.160), signaling a degree of convergent validity. On the other hand, the two specific factors were relatively well-differentiated and the ECV was .583 (ODI scale-level ECV = .440), signaling a degree of discriminant validity. Consistent with these results, the ODI and the DASS-21-D exhibited Pearson and Spearman correlations of .653 and .617, respectively (Table 4).

Table 4.  Pearson and Spearman Correlations Among the Main Study Variables 

Note. N = 386. Pearson correlations are displayed below the diagonal; Spearman correlations are displayed above the diagonal. Correlation coefficients having an absolute value exceeding .096 are statistically significant at p < .05. ODI = Occupational Depression Inventory; DASS-21-D = Depression subscale of the Depression Anxiety Stress Scales-21; M = mean; SD = standard deviation. Sex was coded “0” for women and “1” for men.

Women tended to report more symptoms of occupational depression than men and older individuals tended to report fewer symptoms of occupational depression than their younger counterparts (Table 4). We found occupational depression to correlate substantially, and in the expected direction, with objective cognitive performance and each of the work-life characteristics under examination, all ps < .001. More specifically, occupational depression correlated positively with interpersonal conflict at work, job incivility, unreasonable work tasks, unnecessary work tasks, and work overload (rs ranging from .392 to .555; ρs ranging from .381 to .588) and negatively with social support at work, job autonomy, skill development, job recognition, job meaningfulness, and cognitive reflection (rs ranging from -.196 to -.471, ρs ranging from .198 to -.469). When corrected for measurement error within an SEM framework, the correlation between occupational depression and cognitive reflection reached -.275, RMSEA = .030, CFI = .997, TLI = .996, SRMR = .052, χ2(89) = 120.578 (Supplementary Material 5).

Discussion

In this study, we translated the ODI into Spanish using a back-translation method and examined the structural and psychometric properties of the instrument's Spanish version. The characteristics of the ODI were investigated based on advanced statistical techniques, including ESEM bifactor analysis and Mokken scale analysis. Our findings suggest that the Spanish version of the ODI has excellent structural and psychometric properties and behaves as satisfactorily as the (previously developed) English and French versions of the instrument.

Main Findings

We found the Spanish version of the ODI to meet the requirements for essential unidimensionality, consistent with previous research on the English and French versions of the instrument (Bianchi & Schonfeld, 2020, 2021a, 2022; Hill et al., 2021). In our ESEM bifactor analysis, about 88% of the common variance extracted was explained by the general factor, a proportion that is highly similar to that reported in Bianchi and Schonfeld's (2020) study (see also Bianchi & Schonfeld, 2021a; Hill et al., 2021). In addition, the general factor accounted for about 95% of the reliable variance in total scores. That the ODI demonstrates essential unidimensionality is noticeable given the instrument's coverage of nine different symptoms. This finding speaks to the unity of the phenomenon of (occupational) depression (Rodriguez et al., 2016). Measurement invariance held across our sample and the three samples used in the original validation study of the ODI conducted by Bianchi and Schonfeld (2020). Such results bode well for the comparability of ODI studies across countries and languages. We also found measurement invariance to hold across sexes within our Spanish sample, suggesting that the ODI had a largely similar structure, or meaning, for men and women.

Mokken scale analysis indicated that the scalability and IIO properties of the ODI were strong and did not reveal any violation of the monotonicity and local independence assumptions. It is noteworthy that such qualities have been rarely observed in health and clinical scales (Meijer & Egberink, 2012), including depression and burnout scales (e.g., Adler et al., 2012; Boothroyd et al., 2019; Jordan et al., 2018; Kliem et al., 2020). Overall, our results suggest that (a) ODI items order individuals in terms of a continuum of latent occupational depression and (b) individuals order ODI items in an accurate fashion. All in all, our results offer a solid basis for the use of ODI's total scores in practice.

Mokken scale analysis identified Item 4—fatigue/loss of energy—as the easiest (or most commonly endorsed) item and Item 9—suicidal ideation—as the most difficult (or least commonly endorsed) item of the ODI. On this basis, Item 9 can be thought of as a sentinel item, in the sense that an individual reporting suicidal ideation is likely to report a host of other symptoms of occupational depression. Importantly, while differences in symptom severity and clinical significance are marked between fatigue/loss of energy and suicidal ideation and can be expected to involve a stable hierarchy in terms of item difficulty, differences in the severity and clinical significance of other symptoms, such as cognitive impairment and psychomotor alterations, may not be as clear-cut. Thus, expecting the item-difficulty hierarchy to be exactly the same across studies, samples, or cultures may not be justified. On a related note, while we examined the IIO property in this study, the issue of whether the IIO assumption is generally realistic for health and clinical scales requires further elucidation (Meijer & Egberink, 2012).

As hypothesized, we found signs of both convergent validity and discriminant validity between the ODI and the DASS-21-D. We expected such a result to emerge because, while the ODI and the DASS-21-D both assess depressive symptoms, the ODI assesses depressive symptoms that individuals specifically attribute to their work whereas the DASS-21-D assesses depressive symptoms independently of any attributed cause. Our findings are consistent with those of (a) Hill et al. (2021), who also used the DASS-21-D to gauge cause-neutral depressive symptoms, and (b) Bianchi and Schonfeld (2020), who relied on the 10-item version of the Center for Epidemiologic Studies Depression scale and the Depression subscale of the Hospital Anxiety and Depression Scale as measures of cause-neutral depressive symptoms.

Speaking to the criterion validity of the ODI, we found the ODI to be negatively and moderately associated with objective cognitive performance. These results are, both in terms of direction and association size, in keeping with those commonly observed in research on clinical depression and neuropsychological functioning (e.g., Rock et al., 2014; Snyder, 2013). Moreover, there is concordance between our results and recent findings by Bianchi and Schonfeld (2022), who examined the link between the ODI and the CRT in a French sample. Interestingly, the raw associations between the two measures were larger in the present study. This difference may be partly due to the fact that we imposed time constraints on the completion of the CRT whereas Bianchi and Schonfeld (2022) did not. Because depressed individuals tend to exhibit slowed processing speed (Snyder, 2013), the CRT was likely more challenging to them in the present study than in Bianchi and Schonfeld's (2022) study. On a more general note, the issue of whether impaired cognitive performance in (occupational) depression is primarily accounted for by genuine functional and structural deficits (e.g., due to cortisol-mediated cell damage) or by lack of motivation vis-à-vis the tasks to be undertaken (loss of motivation is a basic aspect of depressive states) remains to be clarified (Grahek et al., 2019; Scheurich et al., 2008; Tran et al., 2020).

We found additional evidence for criterion validity. The ODI showed moderate to strong associations with various work-life characteristics to which we expected the instrument, as an indicator of job-related distress, to be linked. Consistent with past findings on job-related distress (Bianchi et al., 2021; Guthier et al., 2020; Matthews et al., 2021; Melchior et al., 2007; Niedhammer et al., 2015; Schaufeli et al., 2009; Semmer et al., 2015; Sibeoni et al., 2021), we found occupational depression to correlate with interpersonal conflict at work, job incivility, unreasonable work tasks, unnecessary work tasks, work overload, social support at work, job autonomy, skill development, job recognition, and job meaningfulness. Noting that these work-life characteristics can be conceived of as factors related to job stress—either as stressors or as protective and resilience-promoting factors–, our findings are consistent with the fact that the ODI focuses, by design, on depressive symptoms that are causally attributed to job stress. Our findings are also consistent with the observation that depressive symptoms lead to, or involve, alterations in work-life experiences (Bianchi & da Silva Nogueira, 2019; Bianchi & Laurent, 2018; LeMoult & Gotlib, 2019).

Limitations

Our study has at least four limitations. First, we relied on a convenience sample of self-selected participants. Consequently, the representativeness of our sample vis-à-vis its population of reference (e.g., in terms of basic sociodemographic and health characteristics) is unclear. An implication of this state of affairs is that our estimate of occupational depression prevalence is strictly sample-specific and should not be generalized, for instance, to the Spanish working population. On a related note, the number of male participants in our study was relatively small (n = 113), not only by comparison with the number of female participants but also in itself.

Second, we recruited our Spanish-speaking participants in only one country, Spain. It would have been helpful if we had enrolled Spanish speakers from the Americas, including members of the U.S. Hispanic community. We consulted a Colombian-American physician who was not a contributor to the study to evaluate the usability of the Spanish version of the ODI in Spanish-speaking countries in Latin America. In his judgement, the instrument would work well in those countries.

Third, we assessed work-life characteristics using single-item measures. Multiple-item measures are generally expected to do a better job (Fisher et al., 2016). We note, however, that single-item measures are considerably more valid and reliable than sometimes assumed (Lucas & Donnellan, 2012; Mõttus & Rozgonjuk, 2021; Mõttus et al., 2019). As an illustration, Cheung and Lucas (2014) found single-item measures of life satisfaction to perform quite satisfactorily when compared to multiple-item measures of the construct. Similar observations have been made regarding many other constructs, including organizational constructs such as work/family balance or work role clarity (Fisher et al., 2016). Moreover, single-item measures are able to predict major objective outcomes. Single-item, self-rated measures of general health status, for instance, predict mortality (DeSalvo, 2006).

Fourth, we used a cross-sectional design. Although the choice of a cross-sectional design fit our study's purpose and cross-sectional designs are not nearly as limited as one often presumes (see Spector, 2019), a longitudinal design would have allowed us to examine additional properties of the ODI, such as test-retest reliability or measurement invariance across time. We note that depression scales having comparable architectures have shown test-retest reliability and temporal measurement invariance (Kroenke et al., 2001; Stochl et al., 2020).

Conclusion

The Spanish version of the ODI exhibits excellent structural and psychometric properties. These findings dovetail with those obtained with the English and French versions of the measure. We note that we submitted the ODI to a particularly stringent examination in the present study. Few measures of distress have been subject to such scrutiny. Our approach is in keeping with recommendations for (more) thorough inquiries into psychological scales' structural and psychometric properties (Cortina et al., 2020; Hussey & Hughes, 2020).

This study suggests that occupational health specialists can confidently employ the Spanish version of ODI to identify workers who may need (urgent) help and target distress-generating organizational settings. Because the etiology of (occupational) depression is best understood through the interaction, or relationship, between external conditions and internal dispositions (Bianchi et al., 2017; Grahek et al., 2019; Wichers, 2014), we believe it is incumbent on occupational health specialists to identify factors in the work environment that contribute to depression in workers (e.g., management styles undermining autonomy or setting contradictory and unattainable job objectives) and take steps to remedy those adverse working conditions.

Cite this article as: Bianchi, R., Manzano-García, G., Montañés-Muro, P., Schonfeld, E. A., & Schonfeld, I. S. (2022). Occupational depression in a Spanish-speaking sample: Associations with cognitive performance and work-life characteristics. Journal of Work and Organizational Psychology, 38(1), 59-74. https://doi.org/10.5093/jwop2022a5

1The ODI is available in Bianchi and Schonfeld’s (2020) original article on the instrument and from the website of the Society for Occupational Health Psychology (https://sohp-online.org/resources/research-resources/).

2It is noteworthy that these diagnostic criteria are highly consistent with those found in the latest edition of the International Classification of Diseases. (https://icd.who.int/browse11/l-m/en#/http%3a%2f%2fid.who.int%2ficd%2fentity%2f1563440232).

3With the aim of increasing the local relevance of the item, we referred to euros instead of U.S. dollars, and to a racket rather than a bat. The other items of the test did not require editing.

4Omega is derived from the bifactor model under examination.

5Here, McDonald’s ω was computed based on a one-factor confirmatory factor analytic model (WLSMV estimator).

References

Adler, M., Hetta, J., Isacsson, G., & Brodin, U. (2012). An item response theory evaluation of three depression assessment instruments in a clinical sample. BMC Medical Research Methodology, 12, 84-84. https://doi.org/10.1186/1471-2288-12-84 [ Links ]

Ahola, K., Hakanen, J., Perhoniemi, R., & Mutanen, P. (2014). Relationship between burnout and depressive symptoms: A study using the person-centred approach. Burnout Research, 1(1), 29-37. https://doi.org/10.1016/j.burn.2014.03.003 [ Links ]

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). American Psychiatric Publishing. [ Links ]

American Psychological Association. (2015). Stress in America™: Paying with our health. American Psychological Association. [ Links ]

Baldwin, W. (2000). Information no one else knows: The value of self-report. In A. A. Stone, J. S. Turkkan, C. A. Bachrach, J. B. Jobe, H. S. Kurtzman, & V. S. Cain (Eds.), The science of self-report: Implications for research and practice (pp. 3-7). Lawrence Erlbaum Associates Publishers. [ Links ]

Bianchi, R. (2017). Is the “burnout epidemic” an academic fiction? BMJ: British Medical Journal, 358, j4389. https://www.ncbi.nlm.nih.gov/pubmed/28947602Links ]

Bianchi, R. (2020). Do burnout and depressive symptoms form a single syndrome? Confirmatory factor analysis and exploratory structural equation modeling bifactor analysis. Journal of Psychosomatic Research, 131, Article 109954. https://doi.org/10.1016/j.jpsychores.2020.109954 [ Links ]

Bianchi, R., & da Silva Nogueira, D. (2019). Burnout is associated with a depressive interpretation style. Stress and Health, 35(5), 642-649. https://doi.org/10.1002/smi.2897 [ Links ]

Bianchi, R., & Laurent, E. (2018). Editorial commentary: Burnout in cardiology—going to the heart of the misunderstanding. Trends in Cardiovascular Medicine, 28(1), 8-9. https://doi.org/10.1016/j.tcm.2017.07.008 [ Links ]

Bianchi, R., Laurent, E., Schonfeld, I. S., Verkuilen, J., & Berna, C. (2018). Interpretation bias toward ambiguous information in burnout and depression. Personality and Individual Differences, 135, 216-221. https://doi.org/10.1016/j.paid.2018.07.028 [ Links ]

Bianchi, R., Manzano-García, G., & Rolland, J.-P. (2021). Is burnout primarily linked to work-situated factors? A relative weight analytic study. Frontiers in Psychology (section Organizational Psychology), 11, Article 623912. https://doi.org/10.3389/fpsyg.2020.623912 [ Links ]

Bianchi, R., & Schonfeld, I. S. (2020). The Occupational Depression Inventory: A new tool for clinicians and epidemiologists. Journal of Psychosomatic Research, 138, Article 110249. https://doi.org/10.1016/j.jpsychores.2020.110249 [ Links ]

Bianchi, R., & Schonfeld, I. S. (2021a). Occupational depression, cognitive performance, and task appreciation: A study based on Raven’s Advanced Progressive Matrices. Frontiers in Psychology (section Organizational Psychology), 12, Article 695539. https://doi.org/10.3389/fpsyg.2021.695539 [ Links ]

Bianchi, R., & Schonfeld, I. S. (2021b). The Occupational Depression Inventory—a solution for breaking the impasse of burnout measurement. JAMA Surgery, 156(6), 589-590. https://doi.org/10.1001/jamasurg.2021.0018 [ Links ]

Bianchi, R., & Schonfeld, I. S. (2021c). Who needs to be “burned-out”? Time for a new approach to job-related distress. European Psychiatry, 64(1), e65. https://doi.org/10.1192/j.eurpsy.2021.2240 [ Links ]

Bianchi, R., & Schonfeld, I. S. (2022). Is the Occupational Depression Inventory predictive of cognitive performance? A focus on inhibitory control and effortful reasoning. Personality and Individual Differences, 184, Article 111213. https://doi.org/10.1016/j.paid.2021.111213 [ Links ]

Bianchi, R., Schonfeld, I. S., & Laurent, E. (2017). Burnout or depression: Both individual and social issue. The Lancet, 390(10091), 230. https://doi.org/10.1016/S0140-6736(17)31606-9 [ Links ]

Bianchi, R., Schonfeld, I. S., & Laurent, E. (2019). The trouble with burnout: An update on burnout-depression overlap. American Journal of Psychiatry, 176(1), 79. https://doi.org/10.1176/appi.ajp.2018.18091026 [ Links ]

Bianchi, R., Schonfeld, I. S., & Verkuilen, J. (2020). A five-sample confirmatory factor analytic study of burnout-depression overlap. Journal of Clinical Psychology, 76(4), 801-821. https://doi.org/10.1002/jclp.22927 [ Links ]

Bianchi, R., Verkuilen, J., Schonfeld, I. S., Hakanen, J. J., Jansson-Fröjmark, M., Manzano-García, G., Laurent, E., & Meier, L. L. (2021). Is burnout a depressive condition? A 14-sample meta-analytic and bifactor analytic study. Clinical Psychological Science, 9(4), 579-597. https://doi.org/10.1177/2167702620979597 [ Links ]

Boothroyd, L., Dagnan, D., & Muncer, S. (2019). PHQ-9: One factor or two? Psychiatry Research, 271, 532-534. https://doi.org/10.1016/j.psychres.2018.12.048 [ Links ]

Bowling, A. (2005). Just one question: If one question works, why ask several? Journal of Epidemiology and Community Health, 59(5), 342-345. https://doi.org/10.1136/jech.2004.021204 [ Links ]

Cheung, F., & Lucas, R. E. (2014). Assessing the validity of single-item life satisfaction measures: Results from three large samples. Quality of Life Research, 23(10), 2809-2818. https://doi.org/10.1007/s11136-014-0726-4 [ Links ]

Cortina, J. M., Sheng, Z., Keener, S. K., Keeler, K. R., Grubb, L. K., Schmitt, N., Tonidandel, S., Summerville, K. M., Heggestad, E. D., & Banks, G. C. (2020). From alpha to omega and beyond! A look at the past, present, and (possible) future of psychometric soundness in the Journal of Applied Psychology. Journal of Applied Psychology, 105(12), 1351-1381. https://doi.org/10.1037/apl0000815 [ Links ]

Cox, T., Tisserand, M., & Taris, T. (2005). The conceptualization and measurement of burnout: Questions and directions. Work & Stress, 19(3), 187-191. https://doi.org/10.1080/02678370500387109 [ Links ]

Daza, P., Novy, D. M., Stanley, M. A., & Averill, P. (2002). The Depression Anxiety Stress Scales-21: Spanish translation and validation with a Hispanic sample. Journal of Psychopathology and Behavioral Assessment, 24(3), 195-205. https://doi.org/10.1023/A:1016014818163 [ Links ]

DeSalvo, K. B., Bloser, N., Reynolds, K., He, J., & Muntner, P. (2006). Mortality prediction with a single general self-rated health question. A meta-analysis. Journal of General Internal Medicine, 21(3), 267-275. https://doi.org/10.1111/j.1525-1497.2005.00291.x [ Links ]

Dima, A. L. (2018). Scale validation in applied health research: Tutorial for a 6-step R-based psychometrics protocol. Health Psychology and Behavioral Medicine, 6(1), 136-161. https://doi.org/10.1080/21642850.2018.1472602 [ Links ]

Dohrenwend, B. P. (2000). The role of adversity and stress in psychopathology: Some evidence and its implications for theory and research. Journal of Health and Social Behavior, 41(1), 1-19. https://doi.org/10.2307/2676357 [ Links ]

Dura, J. R., Stukenberg, K. W., & Kiecolt-Glaser, J. K. (1990). Chronic stress and depressive disorders in older adults. Journal of Abnormal Psychology, 99(3), 284-290. https://doi.org/10.1037/0021-843X.99.3.284 [ Links ]

Fisher, G. G., Matthews, R. A., & Gibbons, A. M. (2016). Developing and investigating the use of single-item measures in organizational research. Journal of Occupational Health Psychology, 21(1), 3-23. https://doi.org/10.1037/a0039139 [ Links ]

Frederick, S. (2005). Cognitive reflection and decision making. Journal of Economic Perspectives, 19(4), 25-42. https://doi.org/10.1257/089533005775196732 [ Links ]

Gilbert, P. (2006). Evolution and depression: Issues and implications. Psychological Medicine, 36(3), 287-297. https://doi.org/10.1017/S0033291705006112 [ Links ]

Gonzalez-Mulé, E., & Cockburn, B. S. (2021). This job is (literally) killing me: A moderated-mediated model linking work characteristics to mortality. Journal of Applied Psychology, 106(1), 140-151. https://doi.org/10.1037/apl0000501 [ Links ]

Grahek, I., Shenhav, A., Musslick, S., Krebs, R. M., & Koster, E. H. W. (2019). Motivation and cognitive control in depression. Neuroscience & Biobehavioral Reviews, 102, 371-381. https://doi.org/10.1016/j.neubiorev.2019.04.011 [ Links ]

Guthier, C., Dormann, C., & Voelkle, M. C. (2020). Reciprocal effects between job stressors and burnout: A continuous time meta-analysis of longitudinal studies. Psychological Bulletin, 146(12), 1146-1173. https://doi.org/10.1037/bul0000304 [ Links ]

Haslam, N., Holland, E., & Kuppens, P. (2012). Categories versus dimensions in personality and psychopathology: A quantitative review of taxometric research. Psychological Medicine, 42(5), 903-920. https://doi.org/10.1017/s0033291711001966 [ Links ]

Hassard, J., Teoh, K. R. H., Visockaite, G., Dewe, P., & Cox, T. (2018). The cost of work-related stress to society: A systematic review. Journal of Occupational Health Psychology, 23(1), 1-17. https://doi.org/10.1037/ocp0000069 [ Links ]

Hill, C., de Beer, L. T., & Bianchi, R. (2021). Validation and measurement invariance of the Occupational Depression Inventory in South Africa. PLoS ONE, 16(12), Article e0261271. https://doi.org/10.1371/journal.pone.0261271 [ Links ]

Howard, M. C., Follmer, K. B., Smith, M. B., Tucker, R. P., & Van Zandt, E. C. (2021). Work and suicide: An interdisciplinary systematic literature review. Journal of Organizational Behavior. Advance online publication. https://doi.org/10.1002/job.2519 [ Links ]

Hussey, I., & Hughes, S. (2020). Hidden invalidity among 15 commonly used measures in social and personality psychology. Advances in Methods and Practices in Psychological Science, 3(2), 166-184. https://doi.org/10.1177/2515245919882903 [ Links ]

Jordan, P., Steingen, U., Terschüren, C., & Harth, V. (2018). The Maslach Burnout Inventory: A test dimensionality assessment via item response theory. Testing, Psychometrics, Methodology in Applied Psychology, 25, 101-120. [ Links ]

Kliem, S., Beller, J., Tibubos, A. N., Zenger, M., Schmalbach, B., & Brähler, E. (2020). A reanalysis of the Center for Epidemiological Studies-Depression scale (CES-D) using non-parametric item response theory. Psychiatry Research, 290, Article 113132. https://doi.org/10.1016/j.psychres.2020.113132 [ Links ]

Kotov, R., Krueger, R. F., Watson, D., Achenbach, T. M., Althoff, R. R., Bagby, R. M., Brown, T. A., Carpenter, W. T., Caspi, A., Clark, L. A., Eaton, N. R., Forbes, M. K., Forbush, K. T., Goldberg, D., Hasin, D., Hyman, S. E., Ivanova, M. Y., Lynam, D. R., Markon, K., … Zimmerman, M. (2017). The hierarchical taxonomy of psychopathology (HiTOP): A dimensional alternative to traditional nosologies. Journal of Abnormal Psychology, 126(4), 454-477. https://doi.org/10.1037/abn0000258 [ Links ]

Kroenke, K., Spitzer, R. L., & Williams, J. B. W. (2001). The PHQ-9: Validity of a brief depression severity measure. Journal of General Internal Medicine, 16(9), 606-613. https://doi.org/10.1046/j.1525-1497.2001.016009606.x [ Links ]

LeMoult, J., & Gotlib, I. H. (2019). Depression: A cognitive perspective. Clinical Psychology Review, 69, 51-66. https://doi.org/10.1016/j.cpr.2018.06.008 [ Links ]

Lee, J., Lee, E.-H., & Moon, S. H. (2019). Systematic review of the measurement properties of the Depression Anxiety Stress Scales-21 by applying updated COSMIN methodology. Quality of Life Research, 28(9), 2325-2339. https://doi.org/10.1007/s11136-019-02177-x [ Links ]

Ligtvoet, R., van der Ark, L. A., te Marvelde, J. M., & Sijtsma, K. (2010). Investigating an invariant item ordering for polytomously scored items. Educational and Psychological Measurement, 70(4), 578-595. https://doi.org/10.1177/0013164409355697 [ Links ]

Lucas, R. E., & Donnellan, M. B. (2012). Estimating the reliability of single-item life satisfaction measures: Results from four national panel studies. Social Indicators Research, 105(3), 323-331. https://doi.org/10.1007/s11205-011-9783-z [ Links ]

Marsh, H. W., Morin, A. J., Parker, P. D., & Kaur, G. (2014). Exploratory structural equation modeling: An integration of the best features of exploratory and confirmatory factor analysis. Annual Review of Clinical Psychology, 10, 85-110. https://doi.org/10.1146/annurev-clinpsy-032813-153700 [ Links ]

Matthews, T. A., Robbins, W., Preisig, M., von Känel, R., & Li, J. (2021). Associations of job strain and family strain with risk of major depressive episode: A prospective cohort study in U.S. working men and women. Journal of Psychosomatic Research, 147, Article 110541. https://doi.org/10.1016/j.jpsychores.2021.110541 [ Links ]

Meier, S. T., & Kim, S. (2021). Meta-regression analyses of relationships between burnout and depression with sampling and measurement methodological moderators. Journal of Occupational Health Psychology. Advance online publication. https://doi.org/10.1037/ocp0000273 [ Links ]

Meijer, R. R., & Baneke, J. J. (2004). Analyzing psychopathology items: A case for nonparametric item response theory modeling. Psychological Methods, 9(3), 354-368. https://doi.org/10.1037/1082-989X.9.3.354 [ Links ]

Meijer, R. R., & Egberink, I. J. L. (2012). Investigating invariant item ordering in personality and clinical scales: Some empirical findings and a discussion. Educational and Psychological Measurement, 72(4), 589-607. https://doi.org/10.1177/0013164411429344 [ Links ]

Melchior, M., Caspi, A., Milne, B. J., Danese, A., Poulton, R., & Moffitt, T. E. (2007). Work stress precipitates depression and anxiety in young, working women and men. Psychological Medicine, 37(8), 1119-1129. https://doi.org/10.1017/S0033291707000414 [ Links ]

Mirkovic, D., & Bianchi, R. (2019). Physician burnout: Let’s avoid unsubstantiated claims. Nature Reviews Clinical Oncology, 16(2), 136. https://doi.org/10.1038/s41571-018-0150-8 [ Links ]

Mokken, R. J. (1971). A theory and procedure of scale analysis. De Gruyter Mouton. [ Links ]

Mõttus, R., & Rozgonjuk, D. (2021). Development is in the details: Age differences in the Big Five domains, facets, and nuances. Journal of Personality and Social Psychology, 120(4), 1035-1048. https://doi.org/10.1037/pspp0000276 [ Links ]

Mõttus, R., Sinick, J., Terracciano, A., Hebíková, M., Kandler, C., Ando, J., Mortensen, E. L., Colodro-Conde, L., & Jang, K. L. (2019). Personality characteristics below facets: A replication and meta-analysis of cross-rater agreement, rank-order stability, heritability, and utility of personality nuances. Journal of Personality and Social Psychology, 117(4), e35-e50. https://doi.org/10.1037/pspp0000202 [ Links ]

Muthén, L. K., & Muthén, B. O. (1998-2017). Mplus user’s guide (8th ed.). Muthén & Muthén. [ Links ]

Niedhammer, I., Malard, L., & Chastang, J. F. (2015). Occupational factors and subsequent major depressive and generalized anxiety disorders in the prospective French national SIP study. BMC Public Health, 15, Article 200. https://doi.org/10.1186/s12889-015-1559-y [ Links ]

Otero, I., Salgado, J. F., & Moscoso, S. (2021). Criterion validity of cognitive reflection for predicting job performance and training proficiency: A meta-analysis. Frontiers in Psychology (section Organizational Psychology), 12, Article 695539. https://doi.org/10.3389/fpsyg.2021.668592 [ Links ]

Pickles, A., & Angold, A. (2003). Natural categories or fundamental dimensions: On carving nature at the joints and the rearticulation of psychopathology. Development and Psychopathology, 15(3), 529-551. https://doi.org/10.1017/S0954579403000282 [ Links ]

Primi, C., Morsanyi, K., Chiesi, F., Donati, M. A., & Hamilton, J. (2016). The development and testing of a new version of the Cognitive Reflection Test applying item response theory (IRT). Journal of Behavioral Decision Making, 29(5), 453-469. https://doi.org/10.1002/bdm.1883 [ Links ]

Pryce, C. R., Azzinnari, D., Spinelli, S., Seifritz, E., Tegethoff, M., & Meinlschmidt, G. (2011). Helplessness: A systematic translational review of theory and evidence for its relevance to understanding and treating depression. Pharmacology & Therapeutics, 132(3), 242-267. https://doi.org/10.1016/j.pharmthera.2011.06.006 [ Links ]

Putnick, D. L., & Bornstein, M. H. (2016). Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Developmental Review, 41, 71-90. https://doi.org/10.1016/j.dr.2016.06.004 [ Links ]

R Core Team. (2020). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org/Links ]

Rock, P. L., Roiser, J. P., Riedel, W. J., & Blackwell, A. D. (2014). Cognitive impairment in depression: A systematic review and meta-analysis. Psychological Medicine, 44(10), 2029-2040. https://doi.org/10.1017/s0033291713002535 [ Links ]

Rodriguez, A., Reise, S. P., & Haviland, M. G. (2016). Evaluating bifactor models: Calculating and interpreting statistical indices. Psychological Methods, 21(2), 137-150. https://doi.org/10.1037/met0000045 [ Links ]

Roe, D., Slade, M., & Jones, N. (2022). The utility of patient-reported outcome measures in mental health. World Psychiatry, 21(1), 56-57. https://doi.org/10.1002/wps.20924 [ Links ]

Rotenstein, L. S., Torre, M., Ramos, M. A., Rosales, R. C., Guille, C., Sen, S., & Mata, D. A. (2018). Prevalence of burnout among physicians: A systematic review. JAMA, 320(11), 1131-1150. https://doi.org/10.1001/jama.2018.12777 [ Links ]

Rutkowski, L., & Svetina, D. (2017). Measurement invariance in international surveys: Categorical indicators and fit measure performance. Applied Measurement in Education, 30(1), 39-51. https://doi.org/10.1080/08957347.2016.1243540 [ Links ]

Schaufeli, W. B. (2003). Past performance and future perspectives of burnout research. South African Journal of Industrial Psychology, 29(4), 1-15. https://doi.org/10.4102/sajip.v29i4.127 [ Links ]

Schaufeli, W. B., Bakker, A. B., & Van Rhenen, W. (2009). How changes in job demands and resources predict burnout, work engagement, and sickness absenteeism. Journal of Organizational Behavior, 30(7), 893-917. https://doi.org/10.1002/job.595 [ Links ]

Schaufeli, W. B., & Enzmann, D. (1998). The burnout companion to study and practice: A critical analysis. Taylor & Francis. [ Links ]

Scheurich, A., Fellgiebel, A., Schermuly, I., Bauer, S., Wölfges, R., & Müller, M. J. (2008). Experimental evidence for a motivational origin of cognitive impairment in major depression. Psychological Medicine, 38(2), 237-246. https://doi.org/10.1017/s0033291707002206 [ Links ]

Schonfeld, I. S., & Bianchi, R. (2021). From burnout to occupational depression: Recent developments in research on job-related distress and occupational health. Frontiers in Public Health (section Public Mental Health), 9(1978), Article 796401. https://doi.org/10.3389/fpubh.2021.796401 [ Links ]

Schwenk, T. L., & Gold, K. J. (2018). Physician burnout—a serious symptom, but of what? JAMA, 320(11), 1109-1110. https://doi.org/10.1001/jama.2018.11703 [ Links ]

Semmer, N. K., Jacobshagen, N., Meier, L. L., Elfering, A., Beehr, T. A., Kalin, W., & Tschan, F. (2015). Illegitimate tasks as a source of work stress. Work & Stress, 29(1), 32-56. https://doi.org/10.1080/02678373.2014.1003996 [ Links ]

Shi, D., DiStefano, C., Maydeu-Olivares, A., & Lee, T. (2021). Evaluating SEM model fit with small degrees of freedom. Multivariate Behavioral Research. Advance online publication. https://doi.org/10.1080/00273171.2020.1868965 [ Links ]

Shi, D., Maydeu-Olivares, A., & Rosseel, Y. (2020). Assessing fit in ordinal factor analysis models: SRMR vs. RMSEA. Structural Equation Modeling: A Multidisciplinary Journal, 27(1), 1-15. https://doi.org/10.1080/10705511.2019.1611434 [ Links ]

Sibeoni, J., Bellon-Champel, L., Verneuil, L., Siaugues, C., Revah-Levy, A., & Farges, O. (2021). Workplace environment around physicians’ burnout: A qualitative study in French hospitals. Scandinavian Journal of Work, Environment & Health. Advance online publication. https://doi.org/10.5271/sjweh.3977 [ Links ]

Sijtsma, K., & Molenaar, I. W. (2002). Introduction to nonparametric item response theory. Sage. https://doi.org/10.4135/9781412984676 [ Links ]

Sijtsma, K., & van der Ark, L. A. (2017). A tutorial on how to do a Mokken scale analysis on your test and questionnaire data. British Journal of Mathematical and Statistical Psychology, 70(1), 137-158. https://doi.org/10.1111/bmsp.12078 [ Links ]

SIL International. (2021). Ethnologue: Languages of the world. https://www.ethnologue.com/Links ]

Snyder, H. R. (2013). Major depressive disorder is associated with broad impairments on neuropsychological measures of executive function: A meta-analysis and review. Psychological Bulletin, 139(1), 81-132. https://doi.org/10.1037/a0028727 [ Links ]

Spector, P. E. (2019). Do not cross me: Optimizing the use of cross-sectional designs. Journal of Business and Psychology, 34(2), 125-137. https://doi.org/10.1007/s10869-018-09613-8 [ Links ]

Stochl, J., Fried, E. I., Fritz, J., Croudace, T. J., Russo, D. A., Knight, C., Jones, P. B., & Perez, J. (2020). On dimensionality, measurement invariance, and suitability of sum scores for the PHQ-9 and the GAD-7. Assessment. Advance online publication. https://doi.org/10.1177/1073191120976863 [ Links ]

Stochl, J., Jones, P. B., & Croudace, T. J. (2012). Mokken scale analysis of mental health and well-being questionnaire item responses: A non-parametric IRT method in empirical research for applied health researchers. BMC Medical Research Methodology, 12(1), Article 74. https://doi.org/10.1186/1471-2288-12-74 [ Links ]

Straat, J. H., van der Ark, L. A., & Sijtsma, K. (2016). Using conditional association to identify locally independent item sets. Methodology, 12(4), 117-123. https://doi.org/10.1027/1614-2241/a000115 [ Links ]

Streiner, D. L., Norman, G. R., & Cairney, J. (2015). Health measurement scales: A practical guide to their development and use (5th ed.). Oxford University Press. [ Links ]

Tiesman, H. M., Konda, S., Hartley, D., Chaumont Menéndez, C., Ridenour, M., & Hendricks, S. (2015). Suicide in U.S. workplaces, 2003-2010: A comparison with non-workplace suicides. American Journal of Preventive Medicine, 48(6), 674-682. https://doi.org/10.1016/j.amepre.2014.12.011 [ Links ]

Tran, T., Hagen, A. E. F., Hollenstein, T., & Bowie, C. R. (2020). Physical- and cognitive-effort-based decision-making in depression: Relationships to symptoms and functioning. Clinical Psychological Science, 9(1), 53-67. https://doi.org/10.1177/2167702620949236 [ Links ]

van der Ark, L. A. (2012). New developments in Mokken scale analysis in R. Journal of Statistical Software, 48(5), 1-27. https://doi.org/10.18637/jss.v048.i05 [ Links ]

van Schuur, W. H. (2003). Mokken scale analysis: Between the Guttman scale and parametric item response theory. Political Analysis, 11(2), 139-163. https://doi.org/10.1093/pan/mpg002 [ Links ]

Vinkers, C. H., & Schaafsma, F. G. (2021). Burnout urgently needs robust research. Nature, 592(7853), 188. https://doi.org/10.1038/d41586-021-00896-1 [ Links ]

Wichers, M. (2014). The dynamic nature of depression: A new micro-level perspective of mental disorder that meets current challenges. Psychological Medicine, 44(7), 1349-1360. https://doi.org/10.1017/S0033291713001979 [ Links ]

Willner, P., Scheel-Krüger, J., & Belzung, C. (2013). The neurobiology of depression and antidepressant action. Neuroscience & Biobehavioral Reviews, 37(10), 2331-2371. https://doi.org/10.1016/j.neubiorev.2012.12.007 [ Links ]

Supplementary Material 1

Inventario de Depresión Ocupacional

Instrucciones preliminares para los/as encuestados/as

las siguientes afirmaciones aluden al posible impacto que su trabajo podría tener sobre usted.

Lea cada afirmación e indique con qué frecuencia se ha encontrado con los problemas mencionados en las ÚLTIMAS DOS SEMANAS. Utilice la escala proporcionada aquí para responder:

  • 0 = nunca o casi nunca

  • 1 = sólo unos días

  • 2 = más de la mitad de los días

  • 3 = todos los días o casi todos

Aquí tiene un ejemplo:

«Me he sentido ansioso/a por causa de mi trabajo»

  • Si no se ha sentido ansioso/a por su trabajo, seleccione 0.

  • Si se ha sentido ansioso/a por razones que usted considera QUE NO TIENE RELACIÓN CON SU TRABAJO (problemas personales, conyugales, familiares, de salud, etc.), seleccione igualmente 0.

  • Si se ha sentido ansioso/a pero no sabe por qué, seleccione de nuevo 0.

  • Si usted tiene claro que SU TRABAJO le provoca ansiedad, seleccione 1, 2 o 3 para indicar la frecuencia con la que se ha producido.

Ahora puede contestar al cuestionario.

Si usted se ha identificado con algunos de los problemas sugeridos anteriormente, ¿estos problemas le están haciéndose plantear la posibilidad de dejar su trabajo o puesto actual?

Occupational Depression Inventory: SPSS Syntax for a Provisional Diagnosis of Job-related Depressive Disorder

  1. compute DEP = 0.

  2. do if ODI1 ge 3 or ODI2 ge 3.

  3. count DEP = ODI3 (3)

  4. ODI4 (3)

  5. ODI5 (3)

  6. ODI6 (3)

  7. ODI7 (3)

  8. ODI8 (3)

  9. ODI9 (1,2,3).

  10. end if.

  11. if ODI1 ge 3 DEP = DEP + 1.

  12. if ODI2 ge 3 DEP = DEP + 1.

  13. compute DIAG = 0.

  14. if DEP ge 5 DIAG = 1.

Note. The nine items of the Occupational Depression Inventory are coded ODI1 to ODI9.

Items

  • ODI1: anhedonia

  • ODI2: depressed mood

  • ODI3: sleep alterations

  • ODI4: fatigue/loss of energy

  • ODI5: appetite alterations

  • ODI6: feelings of worthlessness

  • ODI7: cognitive impairment

  • ODI8: psychomotor alterations

  • ODI9: suicidal ideation

Supplementary Material 2

Descriptive Statistics for the Cognitive Reflection Test (CRT)

Note. N = 386. Problem 1 = “Elves and toys”; Problem 2 = “Machines and widgets”; Problem 3 = “Athletes and medals”; Problem 4 = “Lake and lily pads”; Problem 5 = “Bat and ball”; Problem 6 = “Marks in the class.” Mean scores fall between 0 and 1. M = mean; SD = standard deviation; SE = standard error.

Supplementary Material 3

Work -Life Characteristics: Items in Spanish and English

1. Interpersonal conflict at work

Esta semana, he experimentado conflictos en el trabajo (por ejemplo, con compañeros, supervisores, subordinados, clientes).

This week, I have experienced conflicts at work (e.g., with coworkers, supervisors, subordinates, clients).

2. Job incivility

Esta semana, la gente me ha faltado al respeto en el trabajo (por ejemplo, compañeros, supervisores, subordinados, clientes).

This week, people have disrespected me at work (e.g., coworkers, supervisors, subordinates, clients).

3. Unreasonable work tasks

Esta semana, en el trabajo, me pidieron que realizara tareas que, en mi opinión, debería realizar otra persona.

This week, I was asked to perform tasks at work that, in my opinion, should be done by someone else.

4. Unnecessary work tasks

Esta semana, en el trabajo, me pidieron que realizara tareas que no tendría que hacer (o que podría hacer con menos esfuerzo) si las cosas estuvieran mejor organizadas.

This week, I was asked to perform tasks at work that I wouldn't have to do (or could do with less effort) if things were better organized.

5. Work overload

Esta semana la carga de trabajo ha sido excesiva.

This week, my workload has been excessive.

6. Social support at work

Esta semana, he recibido el apoyo que necesitaba en el trabajo (por ejemplo, de mis compañeros, supervisores, subordinados).

This week, I have received the support I needed at work (e.g., from my coworkers, supervisors, subordinates).

7. Job autonomy

Esta semana he podido realizar mi trabajo con la autonomía necesaria.

This week, I had the autonomy I needed to do my work.

8. Skill development

Esta semana mi trabajo me ha permitido aprender cosas nuevas.

This week, I had the opportunity to learn new things through my work.

9. Job recognition

Esta semana la calidad de mi trabajo ha sido halagada.

This week, the quality of my work has been praised.

10. Job meaningfulness

Esta semana sentí que mi trabajo era útil.

This week, I felt the job I did was useful.

Supplementary Material 4

Exploratory Structural Equation Modeling Bifactor Analysis of the Occupational Depression Inventory (ODI) and the Depression subscale of the Depression Anxiety Stress Scales-21 (DASS-21-D)

Note. N = 386. We relied on an oblique target rotation. Bifactor loadings ≥ .30 are italicized. RMSEA = .049; CFI = .994; TLI = .990; WRMR = .518; χ2(75) = 144.487. GF = general factor; SF1 = specific factor with DASS-21-D items as targets; SF2 = specific factor with ODI items as targets; C = communality; ECV = explained common variance; S-ECV = scale-level ECV; I-ECV = item-level ECV.

Supplementary Material 5

Correlation between the Cognitive Reflection (cr) and Occupational Depression (od) Factors. N = 386

Received: November 20, 2021; Accepted: February 07, 2022

Correspondence: renzo.bianchi@unine.ch (R. Bianchi).

Conflict of Interest

The authors of this article declare no conflict of interest.

Creative Commons License This is an Open Access article distributed under the terms of the Creative Commons Attribution-Noncommercial No Derivative License, which permits unrestricted non-commercial use, distribution, and reproduction in any medium provided the original work is properly cited and the work is not changed in any way.