Custom services
text in 
English (pdf)
Article in xml format
Article references
Automatic translation
Send this article by e-mail
Cited by SciELO 
Cited by Google 
Similars in
SciELO 
Similars in Google 
Permalink
Introduction
Resilience is conceptualized as the ability of individuals to recover from stressful events (Smith et al., 2008). Several instruments measure resilience (Windle, Bennett, & Noyes, 2011), but those that are written in Spanish and aimed at young people are scarce.
At University of Guadalajara in Mexico and University of Playa Ancha in Chile, observatories of health and quality of life have been developed in order to design intervention programs with students. Several studies indicate that resilience is an important factor associated with quality of life. Consequently, the development of a demonstrably reliable, valid instrument permitting the evaluation and comparison of the resilience of Mexican and Chilean students would be highly valuable.
The Brief Resilience Scale (BRS) is an instrument designed to measure an individual's ability to recover from adversity. It was created in the United States by Smith et al. (2008) and has been adapted for use in other countries, such as Germany (Leontjevas, de Beek, Lataster, & Jacobs, 2014), Malaysia (Amat, Subhan, Jaafar, Mahmud, & Johari, 2014) and Spain (Rodríguez-Rey, Alonso-Tapia & Hernansaiz-Garrido, 2016). In our initial article on this instrument, we explained in detail the cross-cultural adaptation process for the Mexican and Chilean versions of the instrument (Hidalgo-Rasmussen, Chávez-Flores, Rojas, Vilugrón, 2018), whereas in this second article, we focus on confirmatory factor analysis (CFA) of the adapted Mexican version and cross-validation in a Chilean sample. The analysis of possible idiosyncratic response patterns in different countries and their possible comparability justifies a more extensive and separate proposal.
The BRS consists of six items with five response options that range from “strongly agree” to “strongly disagree” (Smith, et al., 2008). To reduce the influence of acquiescence on the responses, the BRS was constructed as a balanced scale, meaning that three items (1, 3 and 5) directly correspond to the resilience construct -that is, high scores indicate high resilience (e.g., "I tend to recover quickly after having experienced difficult situations")- whereas the other three items (2, 4 and 6) are associated in the opposite direction, with high scores reflecting low resilience (e.g.,"It is difficult for me to recover when something bad happens").
Thus far, Rodríguez-Rey, Alonso-Tapia and Hernansaiz-Garrido (2016) have fit a second-order hierarchical model to a heterogeneous sample of 620 Spanish adults, in which the highest-order factor is a theoretical factor (“resilience”) and the primary factors are factors reflecting the polarity of the items (direct items versus inverse items).
The recommendation to reverse-score some of the items in the scales to avoid biases in acquiescence (i.e., the tendency to respond affirmatively to items on the scale, regardless of their content) may interfere in the evaluation of the latent structure. Several studies have been carried out that show that individual differences in response style due to the presence of both types of items can have systematic effects on the covariance structure and serve as a source of misfit (e.g., Abad, Sorrel, Garcia & Aluja, 2016; Carmines & Zeller, 1979; Danner, Aichholzer & Rammstedt, 2015; Tomás, Galiana, Hontangas, Oliver & Sancho, 2013). Some studies show that this effect occurs in samples of individuals with low education levels, low cognitive levels or low income (Meisenberg & Williams, 2008). In such studies, it has been shown that establishing construct validity is problematic for balanced scales, given that the items are grouped more by the semantic content than by the theoretical construct being evaluated (Benson & Hocevar, 1985).
In the case of BRS results, the proposed second-order model can be interpreted as a result of the combined effect of a structure based on the content of the construct (resilience) superimposed on a factorial structure based on the polarity of the items. Although the second-order model is an option, a first-order one-dimensional model has many benefits over the second-order model in the sense that it allows a simpler interpretation of the relationship between resilience and external variables, particularly when it is necessary to analyze invariance in the measure across different samples.
Recently, some methods have been developed to deal with the problem of acquiescence in the CFA of balanced scales; in the present manuscript, three such methods are compared: the first is the model of a construct with two method factors (Vautier & Pohl, 2009 ) in which each item loads onto the same construct (resilience) and onto one of two factors defined by the polarity of the item (direct vs. inverse); the second is a proposal derived from the work of Tomás et al. (2013), in which we add a factor that aims to assess the effect of the method associated with the inverse items; and the final model is that of Savalei and Falk (2014), who propose adding a method factor that is orthogonal to all the theoretical factors. At present, few studies have investigated the factorial validity of the BRS by identifying the role of acquiescence in a one-dimensional model and the cross-validation of the same models in a sample that comes from a different population. Therefore, the objective of this study is to evaluate the factor structure of the Spanish version of the BRS, taking into account acquiescence, in a large sample of Mexican university students and to cross-validate the same models on Chilean university students using structural equation models.
Various methods are evaluated to identify if the structure found by Rodríguez-Rey, Alonso-Tapia and Hernansaiz-Garrido (2016) is due to acquiescence in the answers.
Method
Participants
This cross-sectional study was carried out in 2016 with two samples, one Mexican and the other Chilean. The Mexican sample consisted of 1572 students recently admitted to a regional university center of a Mexican public university that agreed to participate in the study, in which 60.5% of participants were women and 39.5% were men. The mean age was 18.8 (SD = 1.3). The socio-economic level of the students was best described as lower middle class (64.5%), followed by upper or upper middle class (22.1%) and working class (12.4%). One question asked about the education level of the person who contributed the main income to the house: 48.39% reported that this person had a basic education, 36.73% reported secondary education, and only 14.87% reported higher education. The Chilean sample consisted of 1345 students (61.7% women and 39.3% men) who entered a Chilean public regional university and who agreed to participate. The average age was 19.03 (SD = 1.6). The socioeconomic level was low to medium low in 36.4% of the participants, medium to medium high in 59.2%, and high to very high in 4.4%. In the case of the Chilean students, for the person who contributed the main income, 12.04% had only basic education, 73.53% had a secondary or higher education without finishing and 14.42% had higher or postgraduate education.
Instrument
BRS: Brief Resilience Scale (Smith et al., 2008). The BRS is a self-report scale that consists of 6 items with 5 response options that range from “strongly agree” to “strongly disagree.” A higher score reflects a greater degree of resilience. The original version shows good internal consistency (alphas from .80 to .91) with adequate evidence of convergent and discriminant validity.
Procedure
In both the Mexican and Chilean samples, the students answered the questionnaires online in the computer labs during the course of admission to the university, as part of the study observing the risks and quality of life present at each university. Informed consent was acquired from the students. The project was reviewed and approved by the Bioethics Committees of University of Guadalajara, México and University of Playa Ancha, Chile. The guidelines of the Helsinki Declaration on human research were followed.
Statistical and psychometric analysis
Confirmatory factor analysis (CFA) was performed using IBM AMOS v. 23. All analyzes were performed using the covariance matrix with the maximum likelihood estimation (MLE) method. The following models were adjusted: 1) One-factor model (Resilience); 2) A model with two correlated factors (Resilience and Vulnerability); 3) Second order model of Rodríguez-Rey, Alonso-Tapia and Hernansaiz-Garrido (2016); 4) A general factor and a method factor with inverse items, the factor method aims to assess the effect associated with the inverse items is added to the general resilience factor (Tomás, et al., 2013); 5) One-factor general model and two-method factors, in which two factor method due to the polarity of the items are added, and 6) A general factor and a method factor for acquiescence (Savalei & Falk, 2014) in which a method factor is added orthogonal to all the theoretical factors, in this model the factorial loads of the items are set to 1 in a non-recoded database, which is equivalent to setting the loads to 1 for the direct items and -1 for the inverse items in a recoded database. The factorial loads are fixed for the identification of the model. The parameter of interest and that therefore must be estimated is the variance of the method factor that is an indicator of the size of the bias in the acquiescence. Therefore, this model tries to capture the individual tendencies to use response categories consistently across the items but in an idiosyncratic manner among individuals (Abad, et al., 2016).
The model-fitting process was evaluated by means of the following adjustment indicators: a) Chi square (χ2) and its degrees of freedom (b) the root mean square error of approximation (RMSEA) where the values between .05 and .08 represent an acceptable fit and values less than .05 an excellent fit (Steiger, & Lind, 1980), (c) two robust comparison indices with the null model, namely, NFI (Normed Fit Index), and the CFI (Comparative Fit Index ) (Bentler, 1990), and (d) two absolute goodness of fit indexes, namely, GFI (Goodness of Fit Index) and AGFI (Adjusted Goodness of Fit Index). Where NFI ≥ .95; CFI ≥ .95; GFI ≥ .95 and AGFI ≥ .90 represent an excellent fit (Hu, & Bentler, 1999). Additionally, and given that a sequence of alternative models is presented, the Akaike Information Criteria (AIC, Anderson, & Burnham, 1998) will be used, where the lower values are associated with the most appropriate models. The differences in CFI were also used, a difference of .002 or less is considered to be evidence that the imposition of additional restrictions does not lead to a loss of significant adjustment (Meade, Johnson, & Braddy, 2008).
For reliability analysis, the Cronbach alpha coefficient (α) and the Omega Coefficient (ω) were calculated, the latter from the factorial loads of the best fitting model using the Ventura-Leon & Caycho (2017) procedure.
Results
Comparison between models
Table 1 shows the fit indices of the models analyzed, whose order of presentation is as follows: the more general model of a single dimension (model 1) is presented; then, the models in which two dimensions are considered are presented: resilience and vulnerability (model 2), and this same model, but with a second-order factor (model 3); and finally, the models that deal with acquiescence (models 4, 5 and 6). In that order, it can be observed that the one-dimensional model does not present acceptable indicators of fit for either of the two samples, yielding low values for the goodness of fit indexes (CFI, GFI and AGFI) and high error levels (RMSEA). Models 2 and 3, in which the explanation is conceived from two dimensions (resilience and vulnerability), improved indicators of both the two-factor model, in which there is a greater correlation between the two dimensions for the sample of students from Chile (r Mexico: model3 = -.34; r Chile: model3 = -.55), and of a second-order factor that appears because of the correlation between these dimensions in both samples. In both models and in both samples, the fit indicators are the same-in fact, the Akaike Information Criterion (AIC) is the same-so we can note that there are no significant differences in fit between a first-order model with two correlated factors and a second-order model. When comparing these two models to the different samples, it is important to observe that the fit was much better in the case of the sample of Chilean university students, whereas in the case of Mexican university students, the error rate is unacceptable (RMSEA =. 091, 90% CI: (. 076, .106)).
The three models that have been proposed for the treatment of acquiescence present excellent fit indicators in both samples, especially the one-factor general model and two-method factors that show lower scores in the Akaike Information Criterion (AIC) in both samples (AICMexico: model6 = 60.22, AICChile: model6 = 62.92). If we consider all models, in the sample of Chilean students, there was a lower AIC score for the two-correlated-factors models (AICChile: model2 = 60.661) and the second-order factor (AICChile: model3 = 60.663).
Table 1. Values of goodness of fit indices for Confirmatory Factor Analysis Models
Note: df degrees of freedom, RMSEA the Root mean square error of approximation; 90% CI 90% of Confidence Interval, NFI Normal Fit Index, CFI Comparative Fit Index, GFI Goodness of Fit Index, AGFI Adjusted of Fit Index, AIC Akaike information criterion.** p < .001
Statistics for the model of a general factor and two method factors
Table 2 shows the factorial loads for the general factor, called Resilience, and two factors influenced by the polarity of the items, as well as the variances and the standard error in each dimension. The factorial loads for the general factor were higher in the sample of Chilean university students (λ > .39) than in the sample of Mexican university students (λ > .23), despite the factorial loads for item 2 not being significant. At the same time, the factor loads for the method factors are higher in the sample of Mexican students t-an in the sample of Chilean students. It is expected that the loads for the general factor (Resilience) would be greater than for the method factors; however, for most of the items, such a pattern did not occur. Therefore, acquiescence was present in these samples. The variance was also high for these factors, so it is possible that response styles due to acquiescence affected the factorial model, especially in the sample of Mexican students, where the variance of the inverse items was especially high -much higher than for the factor being directly evaluated (S 2 Mexico_ resilience factor = .215 vs. S 2 Mexico_factor method inverse items = .375).
Analysis of Invariance
Table 3 shows the results of a multivariate invariance analysis for the model of a theoretical factor and two method factors to verify if the factorial structures of the BRS were similar in the two samples studied, thereby implying factorial invariance. In this procedure, global tests are initially carried out on a) the factorial loads of the items and b) the variances of the latent variables; if these comparisons do not yield significant differences, the models are considered to be invariant. In the case where there are significant differences, a) an item-to-item analysis is performed, or b) each of the variances of the latent variables is tested to identify where the differences are taking place, respectively. This procedure starts with a model where the parameters are allowed to be different in both samples (i.e., the unrestricted or augmented model) that is compared with one or more models where the parameters are restricted such that they are the same in both samples (i.e., the restricted or compact model). Once both models have been estimated, the chi-square goodness of fit statistic (χ2) is obtained. Because the compact model is nested in the augmented model, one can compare the change in chi-square (Δχ2) with degrees of freedom equal to the difference of the degrees of freedom of the respective models. In the comparison, a significant result is evidence that equivalence in the parameters is violated. In addition, the criterion of Cheung and Rensvold (2002) can be used, which in nested models is reflected by an increase of more than .01 in the Bentler Comparative Index (ΔCFI) and is an indicator that the model with more restrictions should be rejected. In Table 3, those models in which significant differences were found are shown in gray; these models were therefore considered non-invariant among the samples. It is evident that the two most general models were not invariant, neither with respect to the factorial loads of the items being restricted nor with respect to the variances of the latent variables being restricted. In the specific comparisons, it was identified that items 2 and 4, both inverse items, were not invariant in the samples (these results are supported if the criterion based on the chi-square comparison (χ2) was applied, but not for the ΔCFI criterion). Similarly, the variance of the response-style method factor reflecting the inverse items turned out not to be invariant.
Reliability was reflected for the direct items (Mexico: α = .71, ω = .59, Chile: α = .77, ω = .75) and the inverse items (Mexico: α = .74, ω = .65 Chile: α = .77, ω = .76).
Finally, Table 4 and Table 5 present the variance-covariance matrices for the samples of Mexican and Chilean students, respectively.
Table 3. Goodness of fit indices for the Invariance models for the Confirmatory Factor Analysis.
Note: The marked cells indicate those models in which significant differences were found (p < .05) between the unrestricted model and the restricted model
Discussion
The purpose of this study was to evaluate the factorial structure of the Spanish version of the BRS, taking into account acquiescence, in a large sample of Mexican university students, and to cross-validate the same models on Chilean university students, using structural equation models. Different competitive models were examined to compare those that did not take into account acquiescence (one-dimensional, two-dimensional model and second-order models) versus factorial models that separated the effect of the trait to be measured from the effects of the biases that occur with acquiescence (Condon, Ferrando & Demestre, 2006).
Response styles related to acquiescence were modeled, including additional method factors in the CFA. The inclusion of these factors improved the fit of the model. In the analysis, the models that considered acquiescence were generally found to have better indicators than the three models that did not consider it. In particular, the model of two method factors that were orthogonal to the theoretical factor of resilience yielded the best fit indexes. These results are consistent with other studies that have included method factors (Biderman, Nguyen, Cunningham, & Ghorbani, 2011, Billiet & McClendon, 2000, Danner, et al., 2015).
For the Mexican university students, the best fits of the models that consider acquiescence suggest that in this sample, there was considerable bias due to response style, which influenced the calibration of the items and the scores of the participants (Ferrando & Lorenzo-Seva, 2010; Tomás, et al., 2013; Weijters, Geuens & Shillewaert, 2010a, 2010b). The fit indicators in the two-factor models and the second-order model showed very high error levels (RMSEA > .09) that were smaller in the models that addressed acquiescence. At the same time, the amount of variance due to the method factors was greater than that due to the general Resilience factor, which suggests that a disproportionate use of positive-response selection occurred in that sample (Biderman, et al., 2011).
On the other hand, for the sample of Chilean university students, the two-factor model and the second-order factor model, in which acquiescence was not considered, had fit indicators as good as the model with two method factors. In the analysis, the fit indices were found to be similar among these three models; in the case of the second-order model, the factorial loads for this sample were similar to those found in Spain (Rodríguez-Rey, Alonso-Tapia & Hernansaiz-Garrido, 2016). Even the values of the regression loads between the second-order factor (Resilience) and the first-order factors (direct and inverse items) were better in the Chilean sample than those found in Spain, where the loads were very close to 1.
Conclusions
The design of scales in which direct and inverse items are combined to reduce acquiescence is a common practice in assessment. The main contribution of this work is the comparison of models that consider acquiescence and those that do not. In addition, we compared a Mexican sample with a Chilean sample. We suggest that these results be considered when the results of the BRS are analyzed and when studies using CFA are carried out on instruments that are susceptible to acquiescent response styles.
For cross-validation, a multigroup invariance analysis was performed, which showed that the two samples studied had different factorial structures for both the factorial load model, finding differences in the factor loads of two inverse items (items 2 and 4), and for the variance model of the latent variables, finding differences in the variance of the factor associated with the negative items. Such differences can occur when comparing countries, especially in contexts where the evaluation has no consequences and where the respondents have little motivation to perform the evaluation. A possible line of research suggests that in measures that are carried out at a global or inter-country level, it has been found that extreme or acquiescent response styles are observed in countries with low levels of education and income. Particularly, acquiescence is commonly observed in countries with high levels of corruption (Meisenberg & Williams, 2008).
Finally, we must note that even though the results of the study implicate inverse-item method effects on the factorial structure of the scale, these effects also have implications in the estimation of scores in the latent factors. It is important to remember that from the point of view of CFA, fitting of a model usually occurs in two phases: the calibration of the items and the estimation of the scores. It is known that the acquiescent response style can generate bias in the process of model fitting and calibration of the items and therefore affect calculations in the score-estimation phase (Ferrando & Lorenzo-Seva, 2010). In the latent-score estimation phase, the parameters are fixed to the values estimated in the first phase and are used to calculate the trait levels for each person. There are several procedures to calculate the level of trait, but research suggests that Bayesian estimation has advantages over other procedures (Ferrando & Lorenzo-Seva, 2016). Based on these estimates, it would be possible to study the impact of acquiescence on the precision with which the latent factor related to resilience is estimated as well as study the reliability and validity of the scores free from the bias of the acquiescent response styles. At the same time, we can move forward on a line of research in which we seek to identify those variables related to this effect in different populations.
