SciELO - Scientific Electronic Library Online

 
vol.36 número1Desarrollo y validación del Inventario de Estilos de Enseñanza para Educación Superior (IEE)Actividades sexuales online en estudiantes universitarios: relación con la satisfacción sexual índice de autoresíndice de materiabúsqueda de artículos
Home Pagelista alfabética de revistas  

Servicios Personalizados

Revista

Articulo

Indicadores

Links relacionados

  • En proceso de indezaciónCitado por Google
  • No hay articulos similaresSimilares en SciELO
  • En proceso de indezaciónSimilares en Google

Compartir


Anales de Psicología

versión On-line ISSN 1695-2294versión impresa ISSN 0212-9728

Anal. Psicol. vol.36 no.1 Murcia ene./abr. 2020  Epub 07-Dic-2020

https://dx.doi.org/10.6018/analesps.36.1.341001 

Developmental and Educational Psychology

Development and validation of the School Climate Questionnaire for Secondary and High School Teachers (SCQ-SHST)

Desarrollo y validación del Cuestionario de Clima Escolar para Profesores de Secundaria y Bachillerato (CES-PSB)

Ana Quijada1  *  , Miguel A Ruiz1  , Juan A Huertas1  , Jesús Alonso-Tapia1 

1Universidad Autónoma de Madrid (Spain)

Abstract:

The present study focuses on the development and initial validation of a measure of school climate based on teachers’ perceptions. The measure was developed to integrate those aspects of the school climate highlighted in the existing literature and that proved relevant for teachers’ efficacy, including the school goal structure. Exploratory and confirmatory factor analyses were conducted on a sample of 581 high school teachers. Results showed that a seven-correlated factor structure better represented the data, in comparison with a second-order model and a bifactor model. Structural equation modeling techniques provided evidence of the concurrent validity through correlations with a criterion measure of teacher job satisfaction. School goal structure proved to be an important aspect of school climate substantially related to other school climate factors such as school management quality and relationships among teachers. These findings provided some empirical support for the use of the questionnaire in schools, especially for those interested in articulating interventions aimed at improving the learning environment through the seven school climate dimensions addressed.

Keywords: School climate; School goal structure; Teacher perceptions; Teacher satisfaction; School climate assessment

RESUMEN

Resumen: El presente estudio se centra en el desarrollo y validación inicial de una medida del clima escolar basada en las percepciones de los profesores. La medida fue desarrollada para integrar aquellos aspectos del clima escolar destacados en la literatura existente, incluyendo la estructura de metas escolar. Se realizaron análisis factoriales exploratorios y confirmatorios con una muestra de 581 profesores de secundaria y bachillerato. El uso de modelos de ecuaciones estructurales proporcionó evidencia sobre la validez concurrente a través de correlaciones con una medida criterio sobre la satisfacción laboral de los profesores. La estructura de metas escolar demostró ser un aspecto relevante para definir el clima escolar relacionándose sustancialmente con otros aspectos del mismo tales como la calidad del liderazgo o el apoyo entre profesores. Estos resultados proporcionaron evidencias empíricas a favor del uso del cuestionario propuesto entre las escuelas, especialmente en aquellas interesadas en articular intervenciones encaminadas a mejorar el entorno de aprendizaje a través de las siete dimensiones del clima escolar abordadas.

Palabras clave: Clima escolar; Estructura de metas escolar; Percepciones de los profesores; Satisfacción docente; Evaluación del clima escolar

Introduction

School climate has been widely studied over the past decades since it is considered an important indicator of educational quality. Existing literature has documented the relation of school climate with student academic, behavioral and psychological outcomes. For instance, it has been found that student perceptions of school climate are linked to academic achievement (Finnan, Schnepel, & Anderson, 2003), school absenteeism (Gottfredson, Gottfredson, Payne, & Gottfredson, 2005) and peer harassment (Klein, Cornell, & Konold, 2012). Likewise, teacher perceptions of school climate are related to job satisfaction and self-efficacy (Collie, Shapka, & Perry, 2012), experienced burnout (Grayson & Alvarez, 2008), and teacher retention rates (Wynn, Carboni, & Patall, 2007). Thus, it is not surprising that a significant amount of school reform initiatives are focused on improving school climate (Durlak, Weissberg, Dymnicki, Taylor, & Schellinger, 2011).

According to the National School Climate Council (2007), school climate is based on “patterns of people’s experiences of school life and reflects norms, goals, values, interpersonal relationships, teaching and learn ing practices, and organizational structures” (p.4). However, there is no universal definition and, often, these definitions are so broad that they have led to inconsistencies when it comes to measuring this construct (Thapa, Cohen, Guffey, & Higgins-D’Alessandro, 2013). Moreover, most of the school climate measures were initially developed to assess student perceptions and not validated surveys continue to be used by schools (Cohen, Mccabe, Michelli, & Pickeral, 2009; Olsen, Preston, Algozzine, Algozzine, & Cusumano, 2017). Therefore, research efforts aimed at measuring school climate and developing useful and reliable tools are still needed.

School Climate: Dimensions and Measures

The different definitions that have been used to describe the school climate have led to a lack of theoretical coherence in many school climate measures and a lack of consensus regarding the dimensions that should be regularly measured (Thapa et al., 2013; Clifford, Menon, Gangi, Condon, & Hornung, 2012). There are several measures that cover very different dimensions of the school climate. For example, some surveys like the Comprehensive School Climate Inventory (CSCI; National School Climate Center, 2002) offer a comprehensive view of school climate addressing the appearance and physical environment, faculty relations, student interactions, leadership, discipline environment, learning environment, attitude and culture, and school-community relationship. Other surveys, like the California School Climate Health, and Learning Survey (CAL-SCHLS; WestEd, 2014), offer a school climate view based on school discipline, drug use, victimization and physical safety. Given the heterogeneity of the dimensions evaluated, recent reviews (Thapa et al., 2013; Wang & Degol, 2015) have summarized four domains commonly addressed in school climate research: a) Physical and socio-emotional safety, supported by rules and norms; b) Relationships between various groups (e.g., teacher‒student and student‒student relationships); c) Conception of teaching-learning processes; and, d) Institutional environment, defined by physical environment and educational resources. However, a great deal of school climate measures target student perceptions (for a school climate surveys review see Olsen et al., 2017).

According to recent research, student perceptions of school climate may be based on what occurs in smaller ecologies in the school, such as the classroom (Hung, Luebbe & Flaspohller, 2014; Mitchell & Bradshaw, 2013), which is where students spend most of the time. In addition, certain aspects that comprise school climate (e.g., leadership or quality of relationships among teachers) would not be so evident or relevant from student perspectives because of their role within the school (Bandura, 2001; Koth, Bradshaw, & Leaf, 2008). In this context, teacher perspectives become relevant since they perceive and evaluate what happens at school from their own values, interests and motivations, but also play a decisive role in facilitating a supportive classroom climate for students (Künsting, Neuber, & Lipowsky, 2016). Therefore, it is worth asking what are the aspects of school climate that may condition the work and efficacy of teachers. Previous studies have shown that several school climate factors would be related to teacher performance:

  1. Leadership: There is a large body of research that documents principals’ effects on students’ academic achievement (e.g., Louis, Dretzke, & Wahlstrom, 2010). However, research suggests that these effects would be indirect and modulated through the work of others since principal has control over school-level conditions but does not have a direct control over the classroom (Kyriakides, Creemers, Antoniou, Demetriou, & Charalambous, 2015). It has been shown that a leadership in which responsibilities are shared, participation in school decisions is encouraged, and opportunities to improve teaching are provided, is related to a high-quality pedagogy among teachers (Marks & Printy, 2003; Sun & Leithwood, 2015).

  2. Relationships among teachers/teachers-students: Research has shown that teachers who are integrated for the first time and find help from other teachers develop their identity as resilient teachers (Johnson et al., 2015). Likewise, when teachers perceive that their colleagues are willing to assume responsibilities, their involvement in teaching innovation processes increases (Li, Hallinger, & Ko, 2016). Conversely, teachers who perceive a lack of support tend to be victimized (Martinez et al., 2015). It has also been shown that one of the main determinant of student progress is the quality of teacher‒student relationships (Fauth, Decristan, Rieser, Klieme, & Büttner, 2014). Likewise, teachers' perceptions of students influence their effort and the teaching strategies they use (Caprara, Barbaranelli, Steca, & Malone, 2006).

  3. Parent involvement: Research has shown that teachers' perceptions of parental involvement are related to teacher sense of self-efficacy and the expectations they hold about students (Hauser-Cram, Sirin, & Stipek, 2003; Tschannen-Moran & Woolfolk Hoy, 2007). That is, when teachers perceive differences between their own values and those of the students' parents they tend to maintain low expectations for students. Likewise, teachers are frustrated when parents are not involved in students’ learning (Eccles & Harold, 1996).

Apart from these aspects, recent studies have shown that the school goal structure is strongly related to teacher work experiences such as self-efficacy, job satisfaction, emotional exhaustion, time pressure, and motivation to leave the teaching profession (Skaalvik & Skaalvik, 2011; 2017). This fact has important implications for teacher performance and, ultimately, for teaching quality (Klusmann, Kunter, Trautwein, Lüdtke, & Baumert, 2008). However, this aspect has rarely been evaluated from teacher perspective. In fact, school climate measures reviewed to date do not include this aspect despite the impact it seems to have on teacher performance and psychological well-being. Therefore, a better understanding of the key aspects of school climate that are relevant for teachers and that condition their performance is still necessary.

School Goal Structure

Goal theory is commonly used to explain student motivation and academic success (Harackiewicz, Barron, Pintrich, Elliot, & Thrash, 2002). Research has revealed three important aspects in this field. First, students pursue goals of different type. They can face school activities looking to learn because learning generates the experience of being efficient (mastery-goal orientation). On the contrary, they can only worry about results -grades and test scores-, as indicators of ability (performance-goal orientation) or to avoid the emotional experience that follows the failure, especially if it is public (performance-avoidance orientation; Elliot, 2005).

Second, a great deal of research has focused on assessing student perceptions of the classroom goal structure. This concept refers to a set of practices, messages and policies in the classroom that define what constitutes success (Ames, 1992; Midgley et al., 2000). Researchers have identified two main classroom goal structures. A learning goal structure emphasizes understanding and improvement and considers mistakes to be part of the learning process. Conversely, a performance-goal structure emphasizes test scores, public display of grades, and comparison or competition between students (Meece et al., 2006). Students who perceive a learning-goal structure tend to endorse a mastery-goal and use more effective learning strategies such as discriminating relevant information, trying to integrate the new information with the one they know, being more persistent and creative, and taking on more challenging tasks. However, students who perceive a performance-goal structure tend to endorse performance-goal and use surface-level strategies such as rereading the text, memorizing, and guessing (Alonso-Tapia & Pardo, 2006; Ames & Archer, 1988).

Third, students and teachers are also sensitive to the kind of goals that their school conveys (Midgley, Anderman, & Hicks, 1995). The school goal structure is defined by educational goals and values which are strongly emphasized in school and that are made explicit through comments and school practices. For example, comparisons between schools according to test scores, the use of homogeneous ability grouping, or point out students with higher grades as a role model, provide important messages about what constitutes success in a school. However, these goals and values vary from one school to another. Recently, Skaalvik and Skaalvik (2017) found that a learning goal structure is related to teacher job satisfaction and self-efficacy. Conversely, a performance goal structure was associated with an increase in the experience of time pressure, emotional exhaustion and motivation to leave the teaching profession. Similarly, Cho and Shim (2013) showed that teachers who perceived their school as being mastery-oriented were likely to endorse mastery goals for teaching. These results show the key role of school goal structure for teacher performance and psychological well-being. However, this is an underexplored aspect within goal theory and, as these authors point out, instruments that evaluate it in a valid and reliable way are necessary. In addition, the link between school goal structure and other aspects that define school climate, such as school management or quality of teacher relationships, has not been explored in the reviewed works.

In light of the above, the present study had 3 objectives: 1) to develop a measure to evaluate teacher perceptions of school climate defined by those aspects that have proved to be fundamental for their performance; 2) to determine its factor structure and, specially, if the school goal structure is a constituent aspect of school climate; and 3) to study its reliability and concurrent validity. It was hypothesized that the learning-goal structure would be positively related to the rest of school climate components, while the performance-goal structure would be inversely related to these aspects.

Method

Participants

The initial sample consisted of 591 secondary and high school teachers from 92 suburban schools in Madrid, Spain. Participants with missing data (1.69% of the complete sample) were removed, so the final sample was composed of 581 respondents, of which 319 were female (54.9%) and 222 were male (38.2%). Age ranged from 24 to 65 years ( = 41.8, S x =16.3). Teaching experience ranged from 1 to 46 years ( = 16, S x =12). Out of the final sample, 10.2% teachers taught literature, 10.7% mathematics, 10.3% social sciences, 5.9% natural sciences, 4.8% physics/chemistry, 14.8% languages, 3.1% art, 1.2% religion, and 31.7% other subjects. From them, 238 teachers (59%) answered the online version test and 343 (41%) completed the pencil-and-paper questionnaire. It should be noted that 6.9% of the informants did not report their age, sex and years of teaching experience, while 7.4% did not report the subject. For some analyses, the final sample was randomly divided into two samples, the first for developing models in exploratory statistical analysis (n1 = 291, model-derivation sample), and the second for validating such statistical models (n1 = 290, validation sample).

Procedure

Teachers participated voluntarily after their schools received an informative letter explaining the purpose of the present study. Research has shown that paper-and-pencil and Internet data collection methods are generally equivalent (Weigold, Weigold, & Russell, 2013). Thus, the questionnaire was provided in both formats to make it more accessible. Data were collected in the period from October to February of 2015. The University Research Ethics Committee granted approval for the present study.

Instruments

School Climate Questionnaire for Secondary and High School Teachers (SCQ-SHST). A total of 72 items were developed according to the seven primary dimensions of school climate supported by prior research. The number of items varies across dimensions based on the number of components of each of them highlighted in existing literature (e.g., Thapa et al., 2013; Wang & Degol, 2015). Therefore, the 72 items were distributed as follows: School Management Quality (24 items), Teacher-Teacher Relationships (12 items), Mastery-goal Structure (6 items), Performance-Goal Structure (6 items), Teacher-Student Relationships (7 items), Student-Student Relationships (7 items), and Parent Involvement (10 items). Statements were balanced so that there was the same number of positively and negatively worded items for each dimension. They were gauged using a five-point Likert-type scale ranging from 1 (strongly disagree) to 5 (strongly agree). Two experts in educational psychology field reviewed the theoretical relevance of the statements.

Teachers Satisfaction Scale (TSS). A total of 9 items were added as a criterion measure to examine how each one of the aspects of school climate measured contribute to job satisfaction. These items were intended to evaluate two aspects related to teacher satisfaction: 1) satisfaction referred to the way of working in the school (5 items); and 2) satisfaction referred to support and attitudes of students and families (4 items). We decided to evaluate job satisfaction because this variable is closely related to stress, burnout and teacher sense of self-efficacy (e.g., Malinen & Savolainen, 2016). All the items were positively worded and were also gauged using a five-point Likert-type scale ranging from 1 to 5. Table 1 shows item examples from the seven school climate measures and the two teacher satisfaction measures.

Table 1.  Items Examples of the School Climate Questionnaire and Teacher Satisfaction Scale. 

Data Analysis

SCQ-SHST validation. The questionnaire was intended to measure seven dimensions. For each dimension, Parallel Analysis (PA; Horn, 1965) and Exploratory Factor Analysis (EFA) were used with the model-derivation sample to select the items with best psychometric properties. PA was conducted using polychoric correlations and mean eigenvalue criteria as recommended by Garrido, Abad and Ponsoda (2013). EFA was conducted using weighted least squares means and variances adjusted (WLSMV), which is recommended for categorical data (e.g., Muthén, Du Toit, & Spisic, 1997). Items were iteratively removed until the unidimensionality assumption was reached and all the items had factor loadings larger than .30. For this purpose, the highest residual correlation was identified and the item with the smaller loading in this pair was deleted. Then, confirmatory factor analysis (CFA) was used with the second subsample to cross-validate the unidimensional model previously estimated for each dimension. The comparative fit index (CFI) was used to evaluate the model fit considering that conventional cutoff values are ≥ .90 for acceptable fit, and ≥ .95 for good fit (Hu & Bentler, 1999).

In addition, three models were tested and compared using the full sample to select the one that best fits the data. First, we estimated a seven-factor model with correlated domains, since the concept of school climate could imply the interaction among the different components evaluated. Next, a second-order factor model was tested since the relations between the seven components of school climate could be explained by a higher-order factor that corresponds to school climate. Lastly, a bifactor model was tested since the existence of one general factor (i.e., school climate) that affects the variance of items independently of the role played by each specific factor could also be theoretically plausible. These models are illustrated in Figure 1.

Model fit was assessed according to CFI, the Tucker-Lewis Index (TLI), and the root mean square error of approximation (RMSEA). CFI and TLI values of .90 or greater indicate an acceptable fit, while a value of .95 or greater indicates a good fit (Hu & Bentler, 1999). RMSEA values between .05 and .08 represent an acceptable fit, whereas values lower than .05 indicate a good fit (McDonald & Ho, 2002). The Mplus DIFFTEST function was used to conduct comparative tests of nested model fit since traditional chi-square difference tests are not accurate when WLSMV estimation is used. A significant χ2 difference implies that the restrictions added to the nested model make the fit significantly worse and, thus, the first model is retained (Asparouhov & Muthén 2010). The decrease in the χ2/df value (Hair, Black, Babin, & Anderson, 2010), as well as the CFI, TLI and RMSEA recommended cut-off values described above were considered to compare non-nested models.

TSS validation. To determine the number of underlying factors to retain, PA with polychoric correlations and mean eigenvalue criteria were applied. Subsequently, EFA was used to test a model with as many factors as suggested by PA using the model-derivation sample. The WLSMV estimator was used. The cross-validation sample was used to test the model previously estimated and final factor loadings were estimated with the full sample. The model was also assessed according to the CFI, TLI, and RMSEA criteria. Reliability of both questionnaires was evaluated through Cronbach’s alpha coefficient (α). Due to the known limitations presented by this coefficient, the composite reliability (ρc) was additionally obtained (Hair, Hult, Ringle, & Sarstedt, 2013).

Evidence of concurrent validity. A structural equation model (SEM) was tested to examine the relations between school climate dimensions and teacher satisfaction. The model included as many exogenous latent variables as factors present in the validated SCQ-SHST model and as many endogenous latent variables as factors present in the satisfaction model validated for the TSS. For each latent variable, the corresponding observed indicators were included. Standardized regression coefficients were used to examine the relative contribution of exogenous factors to the endogenous factors, and the explained variance of each endogenous latent variable (R 2 ) was used as additional validation criteria. Correlations among the SCQ-SHST factors and the TSS factors were also obtained. CFI, TLI, and RMSEA indices were used to evaluate the model fit.

Figure 1.  Models tested and compared for the School Climate Questionnaire. 

Software. Item selection analysis, EFA, CFA, and SEM analysis were performed using Mplus 7 (Muthén & Muthén, 1998-2012). PA was conducted using the psych package from R (Revelle, 2016). Cronbach’s alpha coefficient and the rate of missing values were computed using SPSS 21 (IBM Corporation, 2012).

Results

SCQ-SHST Validation. Out of 72 items, 9 were removed to preserve the unidimensionality of each dimension. Of these 9 items removed, 5 belonged to School Management Quality construct, 1 to Teacher-Student Relationships, and 3 to Parent Involvement. For each isolated dimension, the unidimensionality assumption was always tenable according to PA with the retained items. The unidimensional solution showed acceptable fit with CFI ≥ .95 in all cases, except for Teacher-Student Relationships (CFI = .89) and Student-Student Relationships dimensions (CFI = .89). All item factor loadings on the isolated unidimensional solutions were statistically significant (p <.001), with average loadings ranging from .67 to .74.

Results of the three models tested are shown in Table 2. As can be seen, the fit was acceptable in all cases. Firstly, the bifactor model and the second-order factor model were compared. The average item loadings for these models are shown in Table 3. Results showed that the factor structure of both models is plausible since the loadings in both the general or second-order factor as well as in each specific factor had a medium-high size. The Learning-Goal Structure showed the highest loading both in the general factor of the bifactor model (.66) and in the second-order factor of the second-order model (.89). When these models were compared using the chi-square difference test, a significant DIFFTEST, χ2 diff(56) = 555.08,p< .0001, suggested that the more restrictive model (the second-order factor model) had a fit significantly worse. Therefore, the bifactor model was retained. Secondly, the bifactor and the seven-correlated factors model were compared. Although the quotient χ2 /df, as well as the remaining fit indexes, were satisfactory for both models, the seven-factor model presented a better fit (χ2 /df= 2.18 < 3; CFI = .94 >.90; TLI = .93 >.90; and RMSEA = .045 <.05). In addition, this model is more parsimonious and provides a simpler interpretation. Thus, the seven-factor model was selected as the final model and was used in subsequent analyses.

Table 2.  Goodness of Fit Indices for Tested Models. 

Table 3.  Bifactor and Second-order Factor Model for the Seven Dimensions of the School Climate Questionnaire: Number of Items and Average Item Loadings on the General or Second-order Factor and Specific Factors. 

The loadings of the seven-factor model estimated with the full sample are shown in Table 4. In the final model, loadings were medium-high sized with values above .48 (average loading was .72). The highest average loadings were found in the School Management Quality, Teacher-Teacher Relationships and Learning-goal Structure dimensions (.74). The lowest average loadings were found in the Teacher-Student Relationships (.67) and Performance-Goal Structure dimension (.69). Student-Student Relationships and Parent Involvement had loadings of .72 on average.

Table 4.  School Climate Questionnaire: CFA Factor Loadings. 

Correlations between the seven dimensions of the SCQ-SHST are shown in Table 5. Moderate to strong correlations across dimensions were found, with coefficients ranging from .39 to .79 (.53 in average), except for Performance-Goal Structure, which was negatively related to all dimensions through correlations that range from -.17 to -.47. As expected, the Learning-Goal Structure was positively related to the other school climate dimensions. In fact, the highest correlation was found between Learning-goal Structure and Teacher Relationships (r = .79). The reliability of the SCQ-SHST was α = .95, ρc = .98, and the reliability of each one of the dimensions are shown in Table 5.

Table 5.  Correlation Coefficients between the SCQ-SHST factors and with the TSS factors, and Reliability Coefficients for SCQ-SHST factors. 

TSS Validation. PA suggested retaining two factors in the model-derivation sample, as expected. Therefore, this model was tested and cross-validated. According to item content, the two factors were labeled as Teacher Satisfaction with School and Teacher Satisfaction with Students and Families. The former was composed of 5 items, and the latter of 4 items. Loadings were high-sized with values above .67. Average loading was .82 for Teacher Satisfaction with School and .76 for Teacher Satisfaction with Students and Families. CFI and TLI values were acceptable (.98 and .97 respectively). RMSEA value was just within the acceptable cutoff (.08) probably due to the large factor loadings size. Recent simulation studies have shown that in the presence of factor loadings above .70 the value of RMSEA tends to increase (McNeish, An, & Hancock, 2017; Savalei, 2012). Correlation between factors was moderate (r = .60). This instrument had a satisfactory reliability α = .87, ρc = .94. Reliability for both Satisfaction with the School (α = .88, ρc = .91) and Satisfaction with Students and Families (α = .78, ρc = .85) was also good.

Figure 2.  Structural equation model tested to examine the relationship between the different dimensions of school climate and teacher satisfaction. 

Evidence of Concurrent Validity. Structural equation modeling analysis assessing concurrent validity of the SCQ-SHST is displayed in Figure 2. Results showed that model fit was good with acceptable values for CFI, TLI, and RMSEA (.92, .92, .049 respectively). The factor that contributed most to both the satisfaction with the school as well as the satisfaction with students and families was the Quality of School Management (β = .75, p < .001). Parental involvement also made an important contribution to satisfaction with students and families (β = .43, p < .001), which was expected. Overall, SCQ-SHST factors explained 88% of teacher satisfaction variance related to school, and 95% of the satisfaction related to students and families. Correlations between the seven SCQ-SHST factors and the two TSS factors are shown in Table 5. All correlations were statistically significant and positive (r = .68 in average) except for the performance-goal structure, whose correlation with both satisfaction with the school and with families and students was lower and negative (r = -.34 and -.35 respectively).

Discussion

Consistent with the aims of this study, results support the proposed seven-factor structure of the SCQ-SHST and present evidence of instrument reliability and concurrent validity. The seven-factor structure was supported by exploratory and confirmatory factor analyses and by model comparison. Several studies have pointed out that bifactor models usually tend to fit better than other competing models (e.g., Reise, 2012). In this study, the bifactor model fitted better than the second-order model, but the seven-factor model reproduced the observed data better and showed a better fit than the remaining models. In addition, the seven-factor model represents a more parsimonious model and provides a simpler and presumably more useful interpretation. For these reasons the seven-factor model was retained. Nevertheless, it does not mean that a higher-order model does not exist or that it cannot be used if necessary both for research and applied purposes since the structure of this model is consistent with the school climate theory proposed and the model fit was satisfactory enough.

The present study support and extend the results of previous studies showing that the school goal structure is a relevant aspect to define school climate and that, therefore, is an aspect to take into account when evaluating such climate. Results of the three models tested support this conclusion. As expected, we found a strong and positive association between learning-goal structure and the school climate dimensions measured, especially with the quality of school management and the relationships among teachers. On the contrary, the performance-goal structure was moderate and negatively related to these aspects. This means that teachers who perceive the emphasis of the school on student learning also tend to perceive other aspects of school climate in positive terms. That is, teachers tend to perceive a school climate characterized by a democratic leadership, colleagues who strive to improve student learning, who propose new initiatives and who support each other instead of criticizing and competing, students interested in learning, and families that support and respect teachers’ work. Conversely, when teachers perceive the emphasis of the school on test scores and high grades, they also perceive an authoritarian leadership, a lack of interest in teachers to improve their work, a teacher-student relationship characterized by the difference of roles, and a lack of support from parents. These results are mostly consistent with previous studies since Skaalvik and Skaalvik (2011) found that teachers feel more identified with a learning-goal structure. That is, they attribute a positive value to this kind of goals. However, Cho and Shim (2013) found that some teachers with a high sense of efficacy consider that performance-approach goals for teaching are also important and that, therefore, constitute a desirable goal.

According to concurrent validity analysis, the quality of school management was the factor that contributed most to explain teacher satisfaction. This finding is consistent with previous studies that point out the strong influence of the principal on the work setting, innovation capacity, and staff motivation (Marks & Printy, 2003; Sun & Leithwood, 2015). The existence of a supportive administration has an important influence teacher commitment to implement the interventions adopted by the school (Pietsch & Tulowitzki, 2017). Likewise, principals control important school-wide conditions, thus, they influence student learning by creating the conditions for better teaching and learning to occur within the school (Clifford, Menon, Gangi, Condon, & Hornung, 2012). For these reasons, it seems appropriate that any reform of the school climate takes into account the evaluation and improvement of aspects related to the quality of leadership, especially when it comes to fostering the commitment and motivation of teachers and, ultimately, student learning.

Interestingly, the relative weight of both learning and performance-goal structure on teacher satisfaction was low. This finding may be due to the kind of items included in the TSS. The items were referred to the management team performance, teacher behaviors, work at school, and families and students support, without making an explicit reference to school goal structure. Despite this, correlations between SCQ-SHST and TSS indicated that the learning-goal structure was strongly and positively associated with teacher satisfaction with the school and with work as a teacher, while the performance-goal structure was moderate and negatively related to it. These findings are in line with research on goal theory since unlike performance-goal structure, the benefits of a learning-goal structure for both teachers and students has been consistently supported (Cho & Shim, 2013; Murayama & Elliot, 2009). According to Skaalvik and Skaalvik (2017), performance-goal structure was associated with an increased experience of time pressure and emotional exhaustion. Therefore, it is possible that pressure to achieve higher grades and the comparison with others that characterizes this kind of goals are one of the reasons why they are associated with low job satisfaction. Instead, when a learning-goal structure is pursued, mistakes are considered part of the learning process and the responsibility of student progress is not completely attributed to the teacher since it is understood that each student has their own learning process. However, this is only a possibility and more research is needed to verify how these goals are perceived by teachers. In any case, it seems appropriate to invest efforts in reflecting and creating a shared vision about the goals and objectives that the school pursues and that will be materialized in the motivation, performance and psychological well-being of teachers and students.

Limitations and future implications

  • The results of this study should be interpreted considering several limitations. First, the SCQ-SHST does not evaluate all dimensions of school climate. This questionnaire does not evaluate those aspect related to Safety and Institutional Environment because these dimensions are less consistently associated with academic outcomes and their effects often disappear when other factors of school climate are controlled (Wang & Degol, 2015). In addition, recently some authors have developed school climate measures that address these aspects from an authoritative approach to school discipline and childrearing (e.g., Bear et al., 2014; Huang et al., 2015), which makes these questionnaires useful for those schools interested in implementing bullying prevention programs. In contrast, the SCQ-SHST provides a perspective of school climate especially useful for those schools interested in articulating interventions aimed at improving the learning environment through the seven dimensions addressed.

  • Second, this study was conducted with a convenience sample of secondary and high school teachers who participated voluntarily. Therefore, a larger and randomly selected sample would be preferable to avoid the limitations related to the generalization of the results.

  • Third, the evidence of the validity of the SCQ-SHST is limited to its factorial validity, face validity, and concurrent validity. Therefore, it is necessary to continue exploring both concurrent and predictive validity of the SCQ-SHST (e.g., through correlations between the SCQ-SHST and a measure not based on self-reports such as teacher retention rate or student achievement) or its psychometric properties with other populations (i.e., cross-cultural validity).

  • Fourth, the final version of the SCQ-SHST was composed of 63 items. Although this is a strength due to the greater scope in the aspects of school climate evaluated in comparison with shorter measures, it may also represent a limitation in some evaluation contexts.

  • Finally, this study evaluates teacher perceptions of the school climate and shows that the aspects included in the SCQ-SHST were relevant from the teacher perspective. Future studies should evaluate the differences between schools in terms of school climate, for which it is necessary to consider the clustering nature of individual participants within schools. Recent studies have shown that multilevel modeling techniques are useful and necessary for this purpose and that, in fact, there seem to be such differences between schools (e.g., Huang et al., 2015).

Conclusions

Results supported the seven-factor structure of the SCQ-SHST and provided some evidence on their suitability to evaluate teacher perceptions of school climate. According to the dimensions evaluated, the SCQ-SHST will be especially useful for those schools interested in articulating interventions focused on improving both teachers performance and learning environment and, ultimately, on improving students academic performance.

References

Alonso-Tapia, J., & Pardo, A. (2006). Assessment of learning environment motivational quality from the point of view of secondary and high school learners. Learning and Instruction, 16, 1-15. doi: 10.1016/j.learninstruc.2006.07.002 [ Links ]

Ames, C. (1992). Achievement goals and the classroom motivational climate. In D.H. Schunk & J.L. Meece (Eds.): Students perceptions in the classroom (pp. 327-348). New York: Lawrence Erlbaum. [ Links ]

Ames, C., & Archer, J. (1988). Achievement goals in the classroom: Students’ learning strategies and motivation processes. Journal of Educational Psychology, 80, 260-267. [ Links ]

Asparouhov, T., & Muthén, B. (2010). Exploratory structural equation modeling. Structural Equation Modeling: A Multidisciplinary Journal, 16, 397-438. doi: 10.1080/10705510903008204 [ Links ]

Bandura, A. (2001). Social cognitive theory: An agentic perspective. Annual Review of Psychology, 52, 1-26. [ Links ]

Bear, G. G., Yang, C., Pell, M., & Gaskins, C. (2014). Validation of a brief measure of teachers’ perceptions of school climate: Relations to student achievement and suspensions. Learning Environments Research, 17, 339-354. doi: 10.1007/s10984-014-9162-1 [ Links ]

Caprara, G. V., Barbaranelli, C., Steca, P., Malone, S. M. (2006). Teachers’ self-efficacy beliefs as determinants of job satisfaction and students’ academic achievement: A study at the school level. Journal of School Psychology, 44(6), 473-490. doi: 10.1016/j.jsp.2006.09.001 [ Links ]

Cho, Y-J, & Shim, S.S. (2013). Predicting teachers' achievement goals for teaching: The role of perceived school goal structure and teachers' sense of efficacy. Teaching and Teacher Education, 32(0), 12-21. doi: 10.1016/j.tate.2012.12.003 [ Links ]

Clifford, M., Menon, R., Gangi, T., Condon, C., & Hornung, K. (2012). Measuring school climate for gauging principal performance: a review of the validity and reliability of publicly accessible measures. Washington, DC: American Institutes for Research. [ Links ]

Cohen, J., McCabe, L., Michelli, N. M., & Pickeral, T. (2009). School climate: research, policy, practice, and teacher education. The Teachers College Record, 111, 180-213. [ Links ]

Collie, R.J., Shapka, J.D., Perry, N. E. (2012). School climate and social-emotional learning: Predicting teacher stress, job satisfaction, and teaching efficacy. Journal of Educational Psychology, 104(4), 1189-1204. doi: 10.1037/a0029356 [ Links ]

Durlak, J. A., Weissberg, R. P., Dymnicki, A. B., Taylor, R. D., & Schellinger, K. B. (2011). The impact of enhancing students’ social and emotional learning: A meta-analysis of school-based universal interventions. Child Development, 82, 405-432. doi: 10.1111/j.1467-8624.2010.01564.x [ Links ]

Eccles, J. S., & Harold, R. D. (1996). Family involvement in children’s and adolescents’ schooling. In A. Booth & J. F. Dunn (Eds.), Family-school links: How do they affect educational outcomes? (pp. 3-34). Mahwah, NJ: Erlbaum. [ Links ]

Elliot, A.J. (2005). A conceptual history of achievement goal construct. In A.J. Elliot & C. Dweck (Eds.): Handbook of competence and motivation (pp. 52-72). New York: Guilford. [ Links ]

Fauth, B., Decristan, J., Rieser, S., Klieme, E., & Büttner, G. (2014). Student ratings of teaching quality in primary school: Dimensions and prediction of student outcomes. Learning & Instruction, 29, 1-9. doi: 10.1016/j.learninstruc.2013.07.001 [ Links ]

Finnan, C., Schnepel, K.C., & Anderson, L.W. (2003). Powerful learning environments: The critical link between school and classroom cultures. Journal of Education for Students Placed at Risk, 8(4), 391-418. doi: 10.1207/S15327671ESPR0804_2 [ Links ]

Garrido, L. E., Abad, F. J., & Ponsoda, V. (2013). A new look at Horn’s parallel analysis with ordinal variables. Psychological Methods, 18(4), 454-474. doi: 10.1037/a0030005 [ Links ]

Gottfredson, G. D., Gottfredson, D. C., Payne, A., & Gottfredson, N. C. (2005). School climate predictors of school disorder: Results from national delinquency prevention in school. Journal of Research in Crime and Delinquency, 42, 421-444. doi: 10.1177/0022427804271931 [ Links ]

Grayson, J. L., & Alvarez, H. K. (2008). School climate factors relating to teacher burnout: A mediator model. Teaching & Teacher Education, 24, 1349-1363. [ Links ]

Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2010). Multivariate data analysis. Upper Saddle River, NJ: Pearson-Prentice Hall. [ Links ]

Hair, J.F., Hult, G.T.M., Ringle, C.M., & Sarstedt, M. (2013). A Primer on Partial Least Squares Structural Equation Modeling (PLS-SEM), Thousand Oaks, United States: SAGE. [ Links ]

Harackiewicz, J. M., Barron, K. E., Pintrich, P. R., Elliot, A. J., & Thrash, T. M. (2002). Revision of achievement goal theory: Necessary and illuminating.Journal of Educational Psychology, 94(3), 638-645. doi: 10.1037/0022-0663.94.3.638 [ Links ]

Hauser-Cram, P., Sirin, S. R., & Stipek, D. (2003). When teachers' and parents' values differ: Teachers' ratings of academic competence in children from low-income families. Journal of Educational Psychology, 95, 813-820. http://dx.doi.org/10.1037/0022-0663.95.4.813 [ Links ]

Horn, J. L. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30, 179-185. [ Links ]

Huang, F., & Cornell, D. (2015). Using multilevel factor analysis with clustered data: Investigating the factor structure of the Positive Values Scale. Journal of Psychoeducational Assessment. Advance online publication. doi: 10.1177/0734282915570278 [ Links ]

Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55. [ Links ]

Hung, A. H., Luebbe, A. M., & Flaspohler, P. D. (2014). Measuring school climate: Factor analysis and relations to emotional problems, conduct problems, and victimization in middle school students. School Mental Health, 7, 105-119. doi: 10.1007/s12310-014-9131-y [ Links ]

IBM Corp. Released 2012. IBM SPSS Statistics for Windows, Version 21.0. Armonk, NY: IBM Corp. [ Links ]

Johnson, B., Down, B., Le Cornu, R., Peters, J., Sullivan, A. M., Pearce, J., & Hunter, J. (2015). Promoting Early Career Teacher Resilience: a socio-cultural and critical guide to action. London: Routledge. [ Links ]

Klein, J., Cornell, D., & Konold, T. (2012). Relationships between bullying, school climate, and student risk behaviors. School Psychology Quarterly, 27(3), 154-169. doi: 10.1037/a0029350 [ Links ]

Koth, C. W., Bradshaw, C. P., & Leaf, P. J. (2008). A multilevel study of predictors of student perceptions of school climate: The effect of classroom-level factors. Journal of Educational Psychology, 100(1), 96-104. [ Links ]

Künsting, J., Neuber, V. & Lipowsky, F. (2016). Teacher Self-Efficacy as a Long-Term Predictor of Instructional Quality in the Classroom. European Journal of Psychology of Education, 31(3), 299-322. [ Links ]

Klusmann, U., Kunter, M., Trautwein, U., Lüdtke, O., & Baumert, J. (2008). Teachers’ occupational well-being and quality of instruction: The important role of self-regulatory patterns.Journal of Educational Psychology, 100(3), 702-715. doi: 10.1037/0022-0663.100.3.702 [ Links ]

Kyriakides, L., Creemers, B.P.M., Antoniou, P., Demetriou, D., & Charalambous, C.Y. (2015). The impact of school policy and stakeholders’ actions on student learning: A longitudinal study. Learning & Instruction, 36, 113-124. doi: 10.1016/j.learninstruc.2015.01.004 [ Links ]

Li, L., Hallinger, P., & Ko, J. (2016). Principal leadership and school capacity effects on teacher learning in Hong Kong. International Journal of Educational Management, 30(1), 76-100. doi: 10.1108/IJEM-03-2014-0035 [ Links ]

Louis, K. S., Dretzke, B. & Wahlstrom, K. (2010). How does leadership affect student achievement? Results from a national US survey. School Effectiveness and School Improvement, 21(3), 315-336. doi: 10.1080/09243453.2010.486586 [ Links ]

Marks, H. M., & Printy, S. M. (2003). Principal Leadership and School Performance: An Integration of Transformational and Instructional Leadership. Educational Administration Quarterly, 39(3), 370-397. doi: 10.1177/0013161X03253412 [ Links ]

Malinen, O. P., & Savolainen, H. (2016). The effect of perceived school climate and teacher efficacy in behavior management on job satisfaction and burnout: A longitudinal study. Teaching and Teacher Education, 10, 144-152. doi: 10.1016/j.tate.2016.08.012 [ Links ]

Martinez, A., McMahon, S.D., Espelage, D., Anderman, E. M., Reddy, L. A., & Sanchez, B. (2015). Teachers’ experiences with multiple victimization: Identifying demographic, cognitive, and contextual correlates. Journal of School Violence, 1-35. doi: 10.1080/15388220.2015.1056879 [ Links ]

McDonald, R. P., & Ho, M.H.R. (2002). Principles and practice in reporting structural equation analyses. Psychological Methods, 7, 64-82. doi: 10.1037/1082-989X.7.1.64 [ Links ]

McNeish, D., An, J., & Hancock, G. R. (2017). The thorny relation between measurement quality and fit index cutoffs in latent variable models. Journal of Personality Assessment. doi: 10.1080/00223891.2017.1281286. [ Links ]

Meece, J.L., Anderman, E.M., & Anderman, L.H. (2006). Classroom goal structure, student motivation and academic achievement. Annual Review of Psychology, 57, 487-503. doi: 10.1146/annurev.psych.56.091103.070258 [ Links ]

Midgley, C., Anderman, E., & Hicks, L. (1995). Differences between elementary and middle school teachers and students: A goal theory approach. Journal of Early Adolescence, 15(1), 90-113. doi: 10.1146/annurev.psych.56.091103.070258 [ Links ]

Mitchell, M. M., & Bradshaw, C. P. (2013). Examining classroom influences on student perceptions of school climate: The role of classroom management and exclusionary discipline strategies. Journal of School Psychology, 51(5), 599-610. doi: 10.1016/j.jsp.2013.05.005 [ Links ]

Murayama, K., & Elliot, A. J. (2009). The joint influence of personal achievement goals and classroom goal structures on achievement-related outcomes. Journal of Educational Psychology, 101, 432-447. doi: 10.1037/a0014221 [ Links ]

Muthén, L. K., & Muthén, B. O. (1998-2012). Mplus user’s guide (7th ed.). Los Angeles, CA: Muthén & Muthén. [ Links ]

Muthén, B., Du Toit, S. H., & Spisic, D. (1997). Robust inference using weighted least squares and quadratic estimating equations in latent variable modeling with categorical and continuous outcomes. Psychometrika, 75, 1-45. [ Links ]

National School Climate Center. (2002). Comprehensive School Climate Inventory. Retrieved from http://www.schoolclimate.org/programs/csci.phpLinks ]

National School Climate Council. (2007). The School Climate Challenge: Narrowing the gap between school climate research and school climate policy, practice guide lines and teacher education policy. Retrieved from http://www.schoolclimate.org/climate/advocacy.phpLinks ]

Olsen, J., Preston, A. I., Algozzine, B., Algozzine, K., & Cusumano, D. (2017). A Review and analysis of selected school climate measures. The Clearing House: A Journal of Educational Strategies, Issues and Ideas. doi: 10.1080/00098655.2017.1385999 [ Links ]

Pietsch, M. & Tulowitzki, T. (2017) Disentangling school leadership and its ties to instructional practices - an empirical comparison of various leadership styles. School Effectiveness and School Improvement, 28(4), 629-649. doi: 10.1080/09243453.2017.1363787 [ Links ]

Reise, S. P. (2012). The rediscovery of bifactor measurement models. Multivariate Behavioral Research, 47, 667-696. doi: 10.1080/00273171.2012.715555 [ Links ]

Revelle, W. (2016) Procedures for personality and psychological research. Evanston, Il: Northwestern University. [ Links ]

Savalei, V. (2012). The Relationship Between Root Mean Square Error of Approximation and Model Misspecification in Confirmatory Factor Analysis Models. Educational and Psychological Measurement, 72(6) 910-932. doi: 10.1177/0013164412452564 [ Links ]

Skaalvik, E. M., & Skaalvik, S. (2011). Teachers’ feeling of belonging, exhaustion, and job satisfaction: the role of school goal structure and value consonance. Anxiety, Stress, & Coping: An International Journal, 24(4), 369-385. doi: 10.1080/10615806.2010.544300 [ Links ]

Skaalvik, E. M., & Skaalvik, S. (2017). Motivated for teaching? Associations with school goal structure, teacher self-efficacy, job satisfaction and emotional exhaustion. Teaching and Teacher Education, 67, 152-160. doi: 10.1016/j.tate.2017.06.006 [ Links ]

Sun, J. , & Leithwood, K. (2015). Direction-setting school leadership practices: a meta-analytical review of evidence about their influence. School Effectiveness and School Improvement, 26(4), 499-523. doi: 10.1080/09243453.2015.1005106 [ Links ]

Thapa, A., Cohen, J., Guffey, S., & Higgins-D’Alessandro, A. (2013). A review of school climate research. Review of Educational Research, 83, 357-385. doi: 10.3102/0034654313483907 [ Links ]

Tschannen-Moran, M., & Woolfolk Hoy, A. (2007). The differential antecedents of self-efficacy beliefs of novice and experienced teachers. Teaching and Teacher Education, 23, 944 -956. [ Links ]

Wang, M.T., & Degol, J. L. (2015). School climate: A review of the construct, measurement, and impact on student outcomes. Educational Psychology Review, 28(2), 315-352. doi: 10.1007/s10648-015-9319-1 [ Links ]

Weigold, A., Weigold, I. K., & Russell, E. J. (2013). Examination of the equivalence of self-report survey-based paper-and-pencil and Internet data collection methods. Psychological Methods, 18, 53-70. doi: 10.1037/a0031607 [ Links ]

WestEd. (2014). California School Climate, Health, and Learning Survey. San Francisco: WestEd. [ Links ]

Wynn, S. R., Carboni, L. W., & Patall, E. A. (2007). Beginning teachers’ perceptions of mentoring, climate, and leadership: Promoting retention through a learning communities perspective. Leadership and Policy in Schools, 6 (3), 209-229. doi: 10.1080/15700760701263790 [ Links ]

Received: March 30, 2019; Revised: June 06, 2019; Accepted: September 12, 2019

Ana Quijada. Universidad Autónoma de Madrid. C/ Iván Pavlov, 6. 28049 Madrid (Spain). E-mail: anna.qdm@gmail.com

Creative Commons License This is an open-access article distributed under the terms of the Creative Commons Attribution License