A psychometric analysis of the Approaches to Teaching Inventory (ATI) and a proposal for a Spanish version (S-ATI-20)

Análisis psicométrico del cuestionario Approaches to Teaching Inventory (ATI) y propuesta en castellano (S-ATI-20)

 

 

Fuensanta Monroy¹, José L. González-Geraldo² y Fuensanta Hernández-Pina¹

¹Universidad de Murcia (Spain)
²Universidad de Castilla-La Mancha (Spain)

Correspondence

 

 

---

ABSTRACT

Background: Recent educational research has explored how university teachers approach their teaching. One of the most widely used instruments is Trigwell and Prosser's (2004) Approaches to Teaching Inventory (ATI). There is, however, scarce research on the Spanish version of this questionnaire. This paper aimed: 1) to analyse the reliability and structural validity of two existing versions of ATI in Spanish language, and 2) to present a proposal for a new Spanish version of ATI, and to measure its reliability and validity.
Method: The samples comprised university and trainee teachers from two Spanish universities.
Results: Reliability coefficients, factor structures, and item loadings of the two existing Spanish versions of ATI were evaluated. Some coincidental outcomes suggested that a revised ATI in Spanish was required, thus a new version (S-ATI-20) was presented and tested on a new sample.
Conclusions: The results suggest questionnaires should be suitably translated and item wording carefully adapted. Factor analyses moderately support a two-scale model, but a lack of correlation between approaches suggests alternative approaches to teaching models might be worth considering. This paper presents preliminary results which should be taken with caution, but S-ATI-20 may serve as a tool for self-reflection in faculty development.

Key words: Approaches to teaching; questionnaire; inventory; teaching process; higher education; university.

---

RESUMEN

Introducción: La investigación educativa de los últimos años ha explorado los enfoques de enseñanza de los profesores universitarios. Uno de los instrumentos más empleados para medir dichos enfoques es el Approaches to Teaching Inventory (ATI) (Trigwell y Prosser, 2004). Los objetivos de esta investigación fueron: a) analizar la fiabilidad y validez de dos versiones en castellano del ATI aplicadas en dos estudios independientes, y b) presentar una propuesta de cuestionario y analizar su consistencia interna y validez.
Método: Las muestras estuvieron constituidas por docentes y profesores en formación de dos universidades españolas.
Resultados: Se detectaron coincidencias en los resultados de ambas versiones en castellano del ATI en cuanto a la fiabilidad, estructura factorial, y cargas de ítems, lo que apuntó a la necesidad de desarrollar una nueva versión (S-ATI-20).
Conclusiones: Los resultados destacan la importancia de traducir los cuestionarios adecuadamente y adaptarlos al contexto. La estructura factorial apoya el modelo de dos escalas del ATI, pero la falta de correlación entre escalas apunta a la posibilidad de considerar otros modelos para explicar los enfoques de enseñanza. Los resultados aquí presentados son preliminares, si bien el S-ATI-20 podría emplearse para la autoreflexión y autoevaluación del profesorado.

Palabras clave: Enfoques de enseñanza; cuestionario; proceso de enseñanza; enseñanza superior; universidad.

---

 

Introduction

For the past twenty years, educational research inspired by the Students' Approaches to Learning (SAL) tradition (Marton, Hounsell, & Entwistle, 1984; Marton & Säljö, 1984) has explored how university teachers approach teaching. Qualitative and quantitative studies have identified two main orientations to teaching: knowledge transmission and learning facilitation (Gow & Kember, 1993; Kember, 1997; Kember & Kwan, 2000; Prosser, Trigwell, & Taylor, 1994). Subsequent investigations developed the Teachers' Approaches to Teaching line of research, and identified a number of approaches to teaching. These may be classified as: a) an Information Transmission/Teacher-Focused (ITTF) approach, characterised by a focus on what teachers do in terms of how they structure, present, manage and transmit the subject matter regardless of what students do; and b) a Conceptual Change/Student-Focused (CCSF) approach, with which teachers attempt to change students' ways of thinking about the subject matter and foster students' construction of their own knowledge (Gow & Kember, 1993; Trigwell, Prosser & Ginns, 2005; Trigwell, Prosser, & Taylor, 1994).

Research evidence suggests that the ways teachers approach their teaching may have important implications for their students' approaches to learning. Thus, a teacher-focused (ITTF) approach to teaching is strongly associated with a surface approach to learning, while a student-focused (CCSF) approach is related to a non-surface and/or deep approach (Gibbs, & Coffey, 2004; Gow & Kember, 1993; Trigwell, Prosser, and Waterhouse, 1999). This association may in turn influence the quality of students' observable learning outcomes.

A key feature of approaches to teaching is context-specificity (Lindblom-Ylänne, Trigwell, Nevgi, & Ashwin, 2006; Stes, Coertjem, & Van Petegem, 2010; Trigwell, & Prosser, 1996b), so a teacher's approach to teaching in one teaching-learning situation may differ in another context. In fact, Trigwell et al. (2005, p. 353) developed the Approaches to Teaching Inventory (ATI) as a "relational indicator of two dimensions or constructs of approaches to teaching." Kember and Kwan (2000, p. 488), on the contrary, argue that teaching approaches are "reasonably stable, so the context dependent aspect is weaker than the predominant effect". Nevertheless, Lindblom-Ylänne et al. (2006) provide empirical evidence for the adoption of different approaches to teaching according to the perception of different teaching contexts, while Postareff, Lindblom-Ylänne, & Nevgi (2007) suggest approaches to teaching change slowly (in the case of pedagogical training programmes).

Using a naturalistic approach and interviews as a source of data, a number of instruments have been developed to measure teachers' teaching orientations or approaches in the tertiary context, such as Gow and Kember's (1993) orientations to teaching questionnaire, and Trigwell and Prosser's (1996a, 2004) Approaches to Teaching Inventory (ATI). More specifically, the latter was developed "to explore the relations between teachers' approaches to teaching and the approaches to learning of students in the classes of those teachers" (2004, p. 416). ATI has been widely administered and undergone several revisions, growing from 16 items (Trigwell & Prosser, 2004) to 22 (Trigwell et al., 2005).

The psychometric properties of the original ATI have been the subject of study. Scale reliability has proved to be moderate-satisfactory (.75 for the CCSF scale, .73 for the ITTF scale (Trigwell & Prosser, 2004); .74 (CCSF), .66 (ITTF) (Prosser & Trigwell, 2006)), while the two-factor structure (CCSF and ITTF) has been confirmed using principal component analyses with varimax rotation (Trigwell & Prosser, 2004) and confirmatory factor analyses (Prosser & Trigwell, 2006). ATI was initially developed as a four-factor model that would reflect teaching intention and strategy making up approaches to teaching. However, there is evidence (Prosser & Trigwell, 2006; Stes, De Maeyer, & Van Petegem, 2010) that it is better understood as a two-dimension instrument similar to Biggs, Kember and Leung's (2001) R-SPQ-2F. Nonetheless, this instrument is not free of criticism, as other studies (see Meyer & Eley, 2006) have severely questioned the psychometric development of ATI, as well as its applicability in activities related to the professionalisation and evaluation of teaching in the tertiary context.

ATI has become increasingly popular in many countries and has been translated into several languages (e.g. Goh, Wong, & Hamzah, 2014, into Malaysian language; Lindblom-Ylänne et al., 2006, into Finnish; Pedrosade-Jesus & da Silva Lopes, 2011; and Rosário et al., 2013, intro Portuguese; Stes, De Maeyer, et al., 2010, into Dutch; Zhang, 2001, into Chinese). Other researchers (Kjellgren et al., 2008), however, overlook giving details of the language ATI is administered in. In Spain ATI has been scarcely used (see Hernández Pina, Maquilón Sánchez, & Monroy Hernández, 2012) and is still largely unknown. Recently, two independent studies administered the ATI in different Spanish contexts in order to measure teaching approaches of university teachers (González Geraldo, 2010) and trainee teachers (Monroy Hernández, 2013).

When questionnaires are used in contexts other than those they were originally developed for, control measures or adaptations are necessary in order to avoid a mere importation of evaluation tools (Buela-Casal, Sierra, Carretero-Dios & De los Santos-Roig, 2002). In such instances, modifications in the wording of items, even when the target language is the same as the original, as well as an adaptation to the context where questionnaires will be administered, are necessary. These are uppermost when instruments are translated into other languages, as translation is not only finding the equivalent word in another language, but is also a sociocultural challenge (Congost Mestre, 2012). Furthermore, key concepts and specific terms need to be explained and contextualised by experts in the field of knowledge, as it is not only a matter of grammar mastery (Oliveira Reis, 2009). In this respect, Prosser and Trigwell (2006) suggest that administrators should modify items in order to reflect a particular context.

Given the popularity of the Approaches to Teaching Inventory, its increasing use in many countries and educational contexts, and the concerns regarding translating questionnaires into other languages, this study analysed the reliability and structural validity results of two existing Spanish versions of ATI, and developed a new version which may be successfully administered to measure Spanish higher education teachers' approaches to teaching.

 

Method

This study analysed two Spanish versions of the Approaches to Teaching Inventory developed by González Geraldo (2010) and Monroy Hernández (2013), hereinafter referred to as Study A and Study B respectively.

Participants

One hundred state university teachers teaching a variety of disciplines took part in Study A during the academic year 2006-2007 (Table 1). The sample in Study B was made up of 239 trainee teachers at a state university enrolled on an initial teacher training programme which offered specialties in various fields of knowledge during the academic year 2012-2013.

 

 

The proposal for a questionnaire derived from this investigation was administered to a sample of 291 trainee teachers (155 females, 136 males) during the academic year 2013-14. Because of the peculiarities of the populations examined and the characteristics of this investigation, there was no control group in either of the studies. A convenience non-probability sampling method was used.

Design

Two studies are reported in this paper: Study A, which used an ex post facto design (Montero & León, 2005), and Study B, which used a pre-experimental design. Bearing in mind that classification systems may have a degree of ambiguity, the design implemented in the present investigation was an instrumental design (Ato, López, y Benavente, 2013, p. 1041). The dependent variable was participants' dominant approach to teaching measured by means of two scales (CCSF and ITTF) and intention and strategy subscales (CCSF-I, CCSF-S, ITTF-I, and ITTF-S), as well as other factors beyond the scope of this study.

Instrument

The two studies reported in this paper administered the Approaches to Teaching Inventory (ATI). This instrument measures participants' student-centred approach to teaching (CCSF) and teacher-centred approach (ITTF), as well as their teaching intention (CCSF-i and ITTF-i subscales) and strategy (CCSF-s and ITTF-s subscales). In the original study, ATI was rated on a 5-point Likert scale, and so was it in Study A (1-Rarely, 2-Sometimes; 3-About half the time; 4-Frequently; 5-Always). Study B, in contrast, used a 4-point scale, as participants expressed the extent to which they agreed with inventory items (1-Totally disagree; 2-Disagree; 3-Agree; 4-Totally agree). All participants scored on ITTF and CCSF.

To date there are two versions of ATI available. Study A administered Trigwell et al. (2005) most recent version (22 items), while in Study B participants completed the 16-item version (Trigwell & Prosser, 2004). In both studies a self-made translation of ATI into Spanish was used. Despite minor differences, both studies followed the same translation procedure: the inventory was translated into Spanish independently by the researcher (a Spanish native speaker) and by two English native speakers not associated with the investigation. The three Spanish translations were compared in order to produce a final version; slight changes were introduced so that the meaning of the original would be kept. For full details of ATI-22 and ATI-16 in Spanish, see González Geraldo (2010) and Monroy Hernández (2013), respectively.

The translation procedure of the final version of ATI proposed in this study (S-ATI-20) was similar to the procedure described above. Minor changes in the wording of some items sought to ensure an understanding of the original items in Spanish and the context-specificity bearing in mind the Spanish educational culture and recent changes introduced in the European Higher Education Area (EHEA). Similar to Stes, De Maeyer, et al. (2010), items were reformulated where ambiguity as to whether they conveyed an ITTF or a CCSF notion was suspected.

Procedure

In this section the procedures carried out in Study A and B, as well as that followed in the present investigation, are reported. Although the questionnaires were administered independently in two universities, the procedure was similar in both cases. The researcher in each study administered the questionnaire, and instructions were given in order to ensure participants would complete ATI anonymously and in a contextualised manner. Participants were informed about the purpose of the investigation, and confidentiality and anonymity were ensured. In Study A participants were asked to respond to ATI-22 bearing in mind their most common way of teaching a particular subject, while in Study B participants were asked to visualise themselves teaching a subject related to the field of study they were being trained for in the initial teacher training programme while completing ATI-16. Questionnaire data in both studies were anonymised by codes prior to statistical analyses.

As to the procedure followed in this investigation, the researchers of each study shared their data with each other and carefully discussed and interpreted their findings. In light of some coincidental outcomes, a proposal for a new Spanish version of ATI was developed (Appendix). In the process some items were discarded and a thorough proofreading of the wording of all items in the new version was carried out, so that ambiguity would be minimised. Thus, the verb "should", which had been formerly translated into Spanish as "debería" (should, ought to, have to, must) was modified, as it may be interpreted as a suggestion, an obligation, a recommendation, or an ideal or hypothetical situation. The new instrument (S-ATI-20) was administered to a new sample following the procedure described above in Study B.

Statistical analyses

Cronbach's alpha was used to calculate the internal consistency of the two Spanish versions of ATI, as well as of the proposal presented here. Structural validity was calculated performing Principal Axis Factoring (PAF) and Maximum Likelihood (ML) analyses with oblique rotation in order to find evidence of Trigwell and Prosser's (2004) postulated two-scale and four-subscale structure. In addition, the factor structure of ATI-22 was examined using Confirmatory Factor Analysis (CFA) .The relation between scales and sub-scales was investigated using Pearson product moment correlations. Statistical analyses were conducted with statistical package SPSS 17 and AMOS 5.

 

Results

Reliability and structural validity results of ATI-22 and ATI-16 are described in this section, followed by preliminary results of the new proposal S-ATI-20.

Former versions of ATI in Spanish

Internal consistency

Cronbach's alpha coefficients were calculated for the ITTF and CCSF scales and subscales of ATI in Study A and B (Table 2). Scale reliability was calculated based on the underlying items of each scale, as well as on the two subscales that make up a scale. CCSF scale reliabilities ranged from .804 to .869 in ATI-22, and from .492 to .505 in ATI-16, while ITTF scale reliabilities were .805-.839 in ATI-22, and .539-.587 in ATI-16. If a particular item was eliminated at scale and subscale level, reliabilities were observed to vary, though only slightly.

Structural validity

Similar to Trigwell et al. (2005) and Stes, De Maeyer, et al. (2010), Study A conducted a confirmatory factor analysis (Table 3) in order to examine the unidimensionality and reliability of ATI-22, as well as the hypothesised latent structure at scale and subscale level observed by Prosser and Trigwell (2006). The results showed a good fit for all dimensions, except the ITTF strategy subscale (ITTF-s), but failed to replicate both models (two-scale and four-scale models proposed by Trigwell and Prosser, 2006), even when the two-factor four-subscale yielded better coefficients (CFI = .989, TLI = .932, RMSEA = .135, SRMR = .016) than the two-scale model (CFI = .741, TLI = .749, RMSEA = .106, SRMR = .112).

An exploratory factor analysis was also conducted in Study A. As Principal Axis Factoring yielded an unclear factor solution, the analysis was forced to two factors (Table 4). Following O'Connor (2000) and prior to the analysis, a Velicer's Minimum Average Partial test (MAP) was run to determine the optimal number of factors that should be extracted and, therefore, check the two-factor structure postulated by Prosser and Trigwell (2006). The two dimensions explained 36.38% of the variance, and all items except two (items 4 and 22) fell into the correct factor, whether CCSF or ITTF. Item 4 (ITTF item) cross-loaded positively and with a similar value on the two factors, while ITTF item 22 failed to load on the appropriate factor (CCSF). Also, item 14 (i.e. 15 in ATI-16) loaded weakly in the CCSF factor.

 

 

In Study B Maximum Likelihood with direct oblimin rotation was calculated on ATI-16 following Brown's (2006) requirements regarding ML in order to check the structure of the instrument. The six factors generated with eigenvalues greater than one explained 35.7% of the variance, and may be interpreted in terms of CCSF or ITTF. Some items, however, did not show a loading high enough to fall into a factor, thus further analysis restricting the solution to only two factors was conducted (Table 4) to ascertain whether items loaded on their corresponding CCSF or ITTF factor. This procedure is relatively common and has been used in previous studies (e.g., Stes, De Maeyer, et al., 2010) to check the two-scale structure posited in the literature. The forced two-factor solution explained 15.85% of the variance, and showed that all items were aligned on one of the two factors except item 1, 9, 15, and 16, which did not reach a minimum loading of .300.

Following similar steps previously noted in the literature, Pearson product moment correlations between the scales and subscales of both versions of ATI were calculated in order to identify a possible relation between approaches, as well as between subscales, respectively, as suggested in the literature. Highly statistically significant positive correlations were found between subscales within each scale in Study A (CCSF subscales: r = .730, p < .001; ITTF subscales: r = .757, p < .001), as well as in Study B (CCSF subscales: r = .327, p <  .000; ITTF subscales: r = .416, p <  .000). Both studies observed a statistically significant negative correlation between the CCSF and ITTF scale (r = -.330, p = .002 in Study A; r = -.110, p = .045, in Study B).

Proposal for a new version of ATI in Spanish (S-ATI-20)

In light of the commonalities in the findings in Study A and B, particularly with regard to reliability coefficients, item loadings, and inconsistency at subscale level, the researchers agreed to pool their findings. The results suggested a reformulation in the wording of some items might be necessary, as well as a modification or elimination of items, particularly if the internal consistency of the inventory was to improve, thus a revised Approaches to Teaching Inventory in Spanish was developed, the S-ATI-20.

Since the version used in Study A (ATI-22) had shown a more coherent factor structure, the researchers agreed to use it as the basis for S-ATI-20. Furthermore, ATI-22 was the most updated version of the inventory; it includes most items of the previous 16-item version, and neglects some items (such as item 1 in ATI-16) which previous studies had revealed as default. S-ATI-20 is similar to ATI-22 but does not include item 14 and 22. Details about the configuration of the instrument are fully explained in the discussion.

S-ATI-20 was administered to a new sample of 291 trainee teachers. Scale reliability of the new version of ATI was .746 (CCSF) and .600 (ITTF). Pearson correlation between the CCSF and ITTF scale showed a non-statistically significant correlation (r = -.025).

Previous studies on the factor structure of ATI have conducted factor analysis with Principal Component Analysis (PCA) and varimax rotation, such as Goh et al. (2014), Stes, De Maeyer, et al. (2010), Trigwell and Prosser (2004), among others. However, bearing in mind Meyer and Eley's (2006) recommendation against "Little Jiffy" (that is, PCA with varimax rotation and the Kaiser criterion, the norm in the literature and by far the most common choice for factor extraction), ML with oblique rotation forced to two factors was performed (Table 5). Furthermore, there is evidence (Costello & Osborne, 2005) that ML produces more generalisable and reproducible results. The results obtained generally matched those obtained in Study A and B, but showed that ITTF items 4, 6, 12, 15 and 18 loaded weakly on their corresponding ITTF factor.

 

 

Discussion

The discussion is structured around the internal consistency and structural validity results of ATI-22 and ATI-16, as well as the development and preliminary findings of S-ATI-20.

Former versions of ATI in Spanish

Following George and Mallery (2003, p. 231) on the acceptability of reliability coefficients, values are to be rated as follows: ">.9, excellent; >.8, good, >.7, acceptable; >.6, questionable; >.5, poor; and <.5, unacceptable," thus scale reliability of ATI-22 (Study A) is good (CCSF: .869; ITTF; .805) and supports Trigwell et al/s (2005) results (.86 for CCSF, and .83 for ITTF). In contrast, scales of ATI-16 (Study B) show poor reliability (CCSF: .505; ITTF: .539), as acknowledged by Monroy Hernández (2013), and do not support Prosser and Trigwell's (2006, p. 410) findings (CCSF: .74; ITTF: .66), which the authors describe as "on the low side of acceptable".

At subscale level, reliability values of ATI-22 are questionable-acceptable, whereas subscale reliabilities of ATI-16 are unacceptable. Monroy Hernández (2013) argues that low scale and subscale reliability of her Spanish version of ATI-16 may be due to some flaws in the initial translation of ATI into Spanish, and recommended further revisions, reason why the present investigation was conducted. Also, she suggests that the amount of teaching experience may have played a role in determining a good comprehension of questionnaire items about teaching. There is evidence of an impact of teaching experience on approaches to teaching (Postareff et al., 2007), so it is likely that teachers in Study A understood ATI as intended, whereas trainee teachers in Study B were still incapable of conceptualising the items of the questionnaire due to lack of solid teaching experience.

As to the factor structure of the questionnaires, the fact that exploratory factor analyses in Study A and B generate more than four components supports the argument against a four-subscale model. Unlike Trigwell et al´s (2005) CFA of ATI-22, which identifies a strong two-factor model for the 22-item ATI (CFI = .95, NNFI = .94, RMSEA = .06, SRMR = .08), the results in Study A (Table 3) show a low consistency of the ITTF strategy subscale. Apart from the (ATI) and a proposalfor a Spanish version (S-ATI-20) 177 fact that TLI and SRMR coefficients are far from ideal, RMSEA is well above the acceptable .08 suggested by Spector (2001), and even further from.05 defended by Browne and Cudeck (1993). Thus, the four-subscale dimensionality initially defended by the original authors, as well as the proposed two-scale models, cannot be fully supported by our data, particularly regarding the ITTF scale. Our results thus lend support to recent investigations in several countries (Goh et al., 2014; Meyer & Eley, 2006; Stes, De Maeyer, et al., 2010). Nevertheless, the two-component solutions when forcing factors do fit the CCSF-ITTF model posited by Prosser and Trigwell (2005), and support previous studies (see Stes, De Maeyer, et al., 2010). Prosser and Trigwell (2006) acknowledge that the inventory should be conceptualised as a two-factor rather than a four-factor instrument, and resolve that approaches to teaching should be described in terms of two main factors which have a hierarchical structure.

The factor solutions obtained in Study A and B (Table 4) show that some items fall into a factor other than the one they were originally intended for. Thus, in Study A ITTF item 4 (It is important to present a lot of facts to students so that they know what they have to learn for this subject) falls into the two constrained factors simultaneously, while item 22 (I present material to enable students to build up an information base in this subject), originally an ITTF descriptor, falls into a CCSF factor. These findings suggest Spanish teachers do not conceive of these two items as straightforward information-transmission statements, and may question the traditional conception that the more information we have, the better learners we are. This idea would be akin to the principles of the Bologna Process and the keystones of the European higher education area in that quantity is no longer the aim to be achieved and the quality of competencies is what now counts.

Interestingly, Study A and B yield similar results regarding the CCSF item on note taking (item 14 in ATI-22, and item 15 in ATI-16: It is better for students in this subject to generate their own notes rather than copy mine). This item cannot be subsumed under any of the two scales, or loads ambiguously (Table 4). This result suggests Spanish teachers do not consider fostering individualised note taking among students to be a feature of a student-centred approach to teaching. In the validation of their Dutch version of ATI, Stes, De Maeyer, et al. (2010, p. 64) also observe that this very item was not properly understood by Flemish university teachers, as they "appear to regard neither making one's own notes nor copying those made by the teacher as something which can straightforwardly be characterized as either student-centred and aimed at conceptual change or teacher-centred and aimed at information transmission." Most recently, a Malaysian version of ATI (Goh et al., 2014) has experienced problems with item 10 of ATI-22, which is also related to note taking (I think an important reason for running teaching sessions in this subject is to give students a good set of notes). These authors suggest that teachers maybe expected to follow this kind of thinking because "teachers in Malaysia have been accustomed to a 'spoon feeding' type of teaching that embraced photocopying notes for students and a drill and practice approach for examination." (p. 24). If encouraging students to take their own notes is not clearly interpreted as a conceptual change feature, it may then be argued that the "spoon feeding type of teaching" is also (and still) widely extended among Spanish university teachers, regardless of the changes introduced following the Bologna Process. We would thus be talking of a Lampedusian Bologna Process (González Geraldo, Trevitt, Carter, & Fazey, 2010), in which changes have been implemented across European educational systems, yet everything has eventually remained unaltered.

Finally, the two studies reported here found a positive correlation between CCSF subscales on the one hand, and ITTF subscales on the other, thus giving evidence of Trig-well and Prosser's (1996a) congruence theory between intention and strategy within each scale, which led them to defend a two-scale model against a subscale interpretation (Prosser & Trigwell, 2006). Furthermore, a negative, low correlation between CCSF and ITTF scales was found, thus lending support to the results obtained by Lindblom-Ylänne et al. (2006), Prosser and Trigwell (2006), and Stes, De Maeyer, et al. (2010). The negative correlation (r = -.26) found by Prosser and Trigwell (2006) may be explained in terms of inventory items describing exclusively the approach they belong to, and not presupposing the inclusiveness principle and hierarchy behind the theory of approaches to teaching postulated by the authors (Trigwell & Prosser, 1996b).

Overall, the fact that a number of items are not consistent with the theoretical model is an indicator that the Approaches to Teaching Inventory in Spanish aimed at university teachers requires revision.

Proposal for a new version of ATI in Spanish (S-ATI-20)

As previous noted, in S-ATI-20 a number of changes were introduced. Item 4 in ATI-22 was reformulated so that the information-transmission notion would be properly conveyed. Item 9 and 16 in the analysis of ATI-16 were default, however, since they did not display any abnormal coefficients in the PAF analysis in Study A (item 9 corresponds to item 8; and item 16 corresponds to item 15 in ATI-22, see Table 4), these two items were kept, but wording was modified. Furthermore, item 14 (in ATI-22) was discarded, as well as item 22. Trigwell et al. (2005) obtained low loadings for item 22 and suggested that "if further testing indicates similar low loadings, a stronger case would exist for [the] exclusion" of this item (p. 357). Interestingly, both items belong to the CCSF and ITTF scale respectively, so the proposed questionnaire S-ATI-20 is well-adjusted in terms of items making up each scale. In eliminating item 22, we followed their recommendation on item elimination, and at the same time tried to avoid reducing the scope of meaning of the two constructs measured by ATI.

As to the rest of the inventory, the wording of all items was thoroughly revised and modified, thus avoiding the ambiguous Spanish verb "debería" (should, ought to, have to, must). This verb might have been interpreted as a suggestion, an obligation, a recommendation, or an ideal or hypothetical situation, which naturally may have caused Study A and B participants some confusion when completing ATI. Nevertheless, rating items on a time-based scale (rarely-always) in Study A may have posed participants some difficulties to clearly understand the verb "debería."

Upon removal and reformulation of items, it was expected that the new version of the inventory would be clearly comprehended and its psychometric properties would improve. Preliminary analyses show that reliability coefficients of the new Spanish version of ATI have dramatically improved, and suggest the instrument should now measure Spanish teachers' approaches to teaching.

As to the factor structure, ML analysis yields moderately satisfactory results. More specifically, five ITTF items (Table 5), namely item 4 (It is important to present a lot of facts to students), item 6 (In this subject I concentrate on covering the information that might be available from key texts and readings), item 12 (I should know the answers to any questions that students may put to me during this subject), item 15 (In this subject my teaching focuses on the good presentation of information to students), and item 18 (My teaching in this subject focuses on delivering what I know to the students) load weakly (below .300) on the ITTF factor. These results suggest that these items are not fully (or even partially) comprehended as information transmission/teacher-focused statements solely. Alternatively, we may argue that some teachers may have understood them as a starting point for more complex teaching processes in which imparting and transmitting information is the first step.

CCSF item 3 (In my interactions with students in this subject I try to develop a conversation with them about the topics we are studying) loads simultaneously on both factors, yet more weakly on the ITTF factor. A possible explanation may be found in the notion conveyed by the word "conversation", translated into Spanish as "conversación." A conversation may be defined as "a talk between two or more people in which thoughts, feelings, and ideas are expressed, questions are asked and answered, or news and information is exchanged" (Cambridge dictionaries online, 2014). Thus, teachers may have interpreted "conversations" as a more superficial task than holding debates and discussions conveyed for instance in CCSF item 8 (In teaching sessions for this subject, I deliberately provoke debate and discussion). Previous studies such as Goh et al. (2014) also observed item 3 loaded onto the information transmission scale, but in their case the authors decided to retain its use in the ITTF factor.

The near-zero correlation between the conceptual change/student-focused scale and the information transmission/teacher-focused scale reveals a lack of association between the two approaches to teaching. This result does not support previous studies (Goh et al., 2014; Lindblom-Ylänne et al., 2006; Meyer & Eley, 2006; Prosser & Trigwell, 2006; Stes, De Maeyer, et al., 2010) and implies that the two scales are non-related descriptors of how university teachers accomplish their teaching. In light of this lack of correlation between variables/factors, orthogonal rotation methods (e.g., varimax) would probably be best in future factor analyses, yet both oblique and orthogonal rotations yield almost identical results if factors are uncorrelated (Costello & Osborne, 2005).

More specifically, our result does not support Prosser and Trigwell's (2006) negative correlation (r = -.26) or Stes, De Maeyer, et al.'s (2010) coefficient (r =-.33), both of which support the argument that ATI measures the CCSF approach independently of the ITTF approach, even though the latter is supposedly embedded in the former (Trigwell & Prosser, 2004). Neither does our result support Goh et al.'s (2014) positive correlation (r = .63), which would imply that the two approaches are related but that ATI does not measure them independently or as two extremes of a continuum. The relation between approaches (reflected in a positive correlation) supports the hierarchical substructure posited by Trigwell et al. (2005), which entails that transmission is seen to be necessary, but rarely sufficient [...]. Transmission elements of the ITTF approach are included in the CCSF approach, but the student-focused element of a CCSF approach is not a part of the ITTF approach. Because of this inclusivity, a CCSF approach is considered to be a more sophisticated or complete approach than the more limiting ITTF approach (p. 352).

Finally, our correlation coefficient does not match Meyer and Eley's (2006) result after performing a ML analysis with oblique rotation on subscale scores, in which they obtain a single factor. These authors argue that Trigwell and Prosser "may have had the wrong a priori model in mind when analysing their data" (p. 646), and conclude that their findings "depict the ATI as conceptually limited to measuring two mutually exclusive approaches within a single dimension of variation, essentially a univariate model, which provided little conceptual modelling power" (p. 643). Opposite signs correlations in various countries suggest the questionnaire should be carefully revised, so that items are clearly and unambiguously formulated, and participants do not mistakenly assume information transmission conceptions when responding student-focused or conceptual change statements.

These findings lead us to three possible models to explain teachers' approaches to teaching:

a) a bipolar continuum, when the ITTF-CCSF correlation is negative. This would imply that the more student-centred an approach is, the less teacher-centred it is, an association which does not really support the hierarchically inclusiveness of approaches to teaching described above;

b) a matryoshka doll model, posited by González Geraldo, Del Rincón & Del Rincón (2011) as a suitable model for explaining students' approaches to learning. This model illustrates approaches unilaterally nested one inside the other, and gives full support to the positive correlation between the two approaches to teaching, whereby ITTF elements are included in the CCSF approach, but not the other way round;

c) a matryoshka doll model, but this time dolls (i.e. approaches) are not embedded, but stand alongside each other depicting the independence of one another. The difference in doll sizes used in this metaphor illustrates that one approach is theoretically more complex than the other. This model would portray a lack of association between approaches to teaching, as shown in this paper, thus a score on one scale is independent of the other. Since participants score on both approaches, and there is evidence of different tendencies in scale scorings depending on the sample (referred to as consonance or dissonance, see Prosser, Ramsden, Trigwell & Martin, 2003), this model would probably be best for portraying the scenario observed in our joint study.

 

Conclusions

This paper reports two studies which analyse the internal consistency and factor structure of two Spanish versions of the Approaches to Teaching Inventory, involving teachers and trainee teachers from two Spanish universities. One of the most noteworthy features is that the two studies reported here converged in some of their results while research was being conducted unknowingly and independently of each other. In this paper a new Spanish version of ATI is proposed (S-ATI-20), and its reliability and structural validity measured. It is the first published investigation which analyses the psychometric properties of ATI within the higher education context in Spain.

Similar to previous studies, the results of this investigation show that the four-factor structure underlying ATI is not replicated in the Spanish versions of the questionnaire. The factor solutions show that components fit better with the two main scales (CCSF and ITTF) originally postulated by Trigwell and Prosser (2004).

The fact that there are coincidences in the items spotted as "ambiguous" or "problematic" in the various versions of ATI analysed in other countries for their particular context supports a premise that questionnaires should be appropriately translated and the wording of items carefully adapted. This is paramount because bringing questionnaires from other countries entails not only translating them accurately, but also fully and deeply grasping nuances and meaning of key concepts (which may convey different concepts in new contexts), as well as subtleties in word order. A single word may have various meanings which may distract or confuse respondents, as shown in this paper, so a straightforward translation may not be enough. A revision of the wording of items in translated questionnaires should also ensure cultural and context-dependent issues are taken into account. The ambiguity posed by item wording, as evidenced above with regard to the verb "debería" in Spanish (and also "should" in English) is what makes us question Goh et al. (2014, p. 19) assertion that "like all other rigorously developed measures, the Approaches to Teaching Inventory items are written in an unambiguous and straightforward language." The different results obtained in the studies reported here give evidence to the fact that some inventory items may be far from being written "in an unambiguous and straightforward language."

In a similar vein, this study also shows that certain teaching-learning activities are context-dependent activities conceptualised differently in different countries. In our particular case, and similar to Stes, De Maeyer, et al.'s (2010) research, note taking was not understood as a conceptual change/student-focused feature. Researchers and administrators should be aware of context-dependence, as it may dramatically affect participants' responses to a questionnaire. Context-specificity sharply contrasts with the assumption of generalisability posed by Trigwell and Prosser in their various studies, and which may have led researchers and administrators of ATI to automatically assume this instrument would measure approaches to teaching across disciplines and cultures. Nonetheless, the authors do acknowledge that "the inventory was developed from a relational study, the results of which are not necessarily transferable to other contexts, and it was specifically designed for a study of approaches to teaching in first year university science classes" (Trigwell & Prosser, 1996a, p. 85). The fact that similar results were obtained with different samples like those used in Study A and B would be a point in favour of the generalisability of ATI.

This study acknowledges a number of limitations which ought to be borne in mind in future research. Firstly, the samples used in each of the two studies are different. While teachers in Study A come from a variety of disciplines, in Study B participants are trainee teachers for whom teaching is not yet an everyday reality. It may be argued that trainee teachers are supposedly not yet fully capable of portraying a realistic picture of teaching, but the fact that the new instrument S-ATI-20 yields satisfactory results in terms of reliability on such a sample, shows that they are, in fact, not so different from regular teachers. At the same time, we are aware of the evidence in the literature that disciplinary backgrounds, as well as teaching experience, may play a role in determining respondents' approaches to teaching. Furthermore, a larger sample and a sample comprising teachers from across fields of knowledge and universities would have been preferable in our preliminary study of S-ATI-20.

Secondly, Trigwell et al. (2005) recommendation regarding modifying the order of items of ATI was not implemented in our first trialling of S-ATI-20. They recommend trialling a different order of the items, particularly so because a response to one item may affect the response to another related item, thus further research may wish to consider presenting inventory items in a different order. This might be done taking item loadings in factor analysis as a criterion for ordering items in such a way that items with higher coefficients would be presented first.

As to the translation procedure, researchers may wish to implement the backtranslation method used in other studies (e.g., Lindblom-Ylänne et al., 2006) when translating surveys into other languages. The procedure described in this paper, although not ideal, did intend to keep the process as objective and neutral as possible with the help of independent native speakers.

It may be argued that both Study A and B should have conducted a CFA in order to confirm the factor structure of ATI. Researchers, however, are often faced with constrains as to performing certain types of tests. While the literature suggests the inclusion of a minimum of 100 participants (or 5-10 cases per parameter) when using Structural Equation Modelling (Hair, Anderson, Tatham, & Black, 1998), Study A decided to use the sample available (n = 100) for both PAF and CFA, while being fully aware of the need to cross-validate these results using different samples (Brown, 2006), as well as other programmes less sensitive to small samples.

Administering perception questionnaires entails great difficulty in monitoring if participants are completing them as intended. Some authors (see Richardson, 2004) question individuals' ability to provide valid reports on their performance in and attitudes towards teaching and learning in a particular context. Also, the extent to which respondents may fake their answers is another issue worth bearing in mind when administering questionnaires. There is evidence (Furnham, 1990) that personality tests, as well as those which measure behaviour patterns, are susceptible to deliberate faking. Other authors (Kirton, 1991), by contrast, differ as to the extent to which participants are indeed capable of faking questionnaires without knowing the factor structure of the instrument itself and what survey-administrators/researchers/employers actually think the right answer should be.

In order to avoid some of these limitations and pitfalls, questionnaire items should be fully contextualised, and qualitative instruments such as interviews may be useful for portraying a clearer picture of participants' responses. In addition, researchers should be appropriately trained in administering, analysing and interpreting data. Making informed choices as to which statistical test to conduct would ensure studies are replicable and of practical value. The results presented in this paper warn us against automatically assuming that measuring approaches to teaching is only a matter of trusting an instrument (psychometrically speaking) which has worked in another context.

In light of the lack of full support for previous studies, the results of S-ATI-20 should be taken with caution, as they are still preliminary results. ATI does not assess general orientations but specific responses to a particular context (Prosser & Trigwell, 2006), which is why this instrument should not be conceptualised as a tool for classifying teachers or for showing a representative picture of teachers in general terms. This is particularly interesting, as approaches to teaching are relational or contextual (Trigwell & Prosser, 2004), which means that the approach adopted by a teacher in one teaching-learning situation may not be the same as in a different context.

In spite of the criticisms and lack of fully satisfactory psychometric properties of S-ATI-20 reported above, this study shows an improved internal consistency of the inventory in Spanish, so a number of implications for teachers and educational authorities may be inferred. S-ATI-20 may well serve as a resource in faculty development and a tool for selfreflection. It would also allow researchers continue exploring teachers' approaches to teaching in our Spanish context with the aim of gathering teachers in focus groups where they may reflect upon and analyse their own teaching performance while sharing experiences with peers. ATI should not be regarded as a magic wand for measuring all possible approaches to teaching, yet it may be used as a trigger for discussion, self-assessment and change, as suggested by Trigwell et al. (2005). Hence, changes in teaching would lead to changes in students' approaches to learning, and thus to higher quality learning outcomes, which would in turn improve the quality of Spanish universities.

 

References

1. Ato, M., López, J. J., y Benavente, A. (2013). Un sistema de clasificación de los diseños de investigación en psicología. Anales de Psicología, 29(3), 1038-1059.         [ Links ]

2. Biggs, J. B., Kember, D., & Leung D. Y. P. (2001). The Revised Two-Factor Study Process Questionnaire: R-SPQ-2F. British Journal of Educational Psychology, 71, 133-149.         [ Links ]

3. Brown, T. A. (2006). Confirmatory factor analysis for applied research. New York: The Guilford Press.         [ Links ]

4. Browne, M. W. & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & J.S. Long (Eds.), Testing structural equation models (pp. 136-162). Thousand Oaks: Sage.         [ Links ]

5. Buela-Casal, G., Sierra, J.C., Carretero-Dios, H., & De los Santos-Roig, M. (2002). Situación actual de la evaluación psicológica en lengua castellana. Papeles del Psicólogo, 83, 27-33.         [ Links ]

6. Cambridge dictionaries online.  (2014). Retrieved from http://dictionary.cambridge.org/.         [ Links ]

7. Congost Mestre, N. (2012). Aspectos socioculturales en la traducción de cuestionarios de salud estadounidenses. Panacea. Revista de Medicina, Lenguaje y Traducción, 13 (35), 91-98.         [ Links ]

8. Costello, A. B., & Osborne, J. (2005). Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis. Practical Assessment Research and Evaluation, 10 (7), 1-9.         [ Links ]

9. Furnham, A. (1990). Faking Personality Questionnaires: Fabricating Different Profiles for Different Purposes. Current Psychology: Research & Reviews, 9(1), 46-55.         [ Links ]

10. George, D., & Mallery, P. (2003). SPSS for Windows step by step: A simple guide and reference, 11.0 update (4th ed.). Boston: Allyn and Bacon.         [ Links ]

11. Gibbs, G., & Coffey, M. (2004). The impact of training of university teachers on their teaching skills, their approach to teaching and the approach to learning of their students. Active learning in Higher Education, 5(1), 87-100.         [ Links ]

12. Goh, P. S. C., Wong K.T., & Hamzah, M. S. G. (2014). The Approaches to Teaching Inventory: A Preliminary Validation of the Malaysian Translation. Australian Journal of Teacher Education, 39(1), 16-26.         [ Links ]

13. González Geraldo, J. L. (2010). Optimización de procesos educativos en el E.E.E.S. (Doctoral Thesis, Universidad de Castilla La Mancha, Spain), Ediciones de la UCLM, ISBN 978-84-8427-771. Retrieved from https://ruidera.uclm.es/xmlui/bitstream/handle/10578/1441/TESIS FINAL ENCUADERNADA.pdf?sequence=1.         [ Links ]

14. González Geraldo, J. L., Del Rincón, B., & Del Rincón, D. (2011). Estructura latente y consistencia interna del R-SPQ-2F. Reinterpretando los enfoques de aprendizaje en el EEES. Revista de Investigación Educativa, 29(2), 277-294.         [ Links ]

15. González Geraldo, J. L., Trevitt, C., Carter, S., & Fazey, J. (2010). Rethinking the Research-Teaching Nexus in Undergraduate Education: Spanish laws pre-and post-Bologna. European Educational Research Journal, 9(1), 81-91.         [ Links ]

16. Gow, L., & Kember, D. (1993). Conceptions of teaching and their relationship to student learning. British Journal of Educational Psychology, 63, 20-33.         [ Links ]

17. Hair, J. F., Anderson, R.., Tatham, R. L., and Black, W. C. (1998). Multivariate data analysis. (5^th ed.). Upper Saddle River, NJ: Prentice-Hall International.         [ Links ]

18. Hernández Pina, F., Maquilón Sánchez, J. J., & Monroy Hernández, F. (2012). Estudio de los enfoques de enseñanza en profesores de educación primaria. Revista de Curriculum y Formación del Profesorado, 16(1), 61-77.         [ Links ]

19. Kember, D. (1997). A reconceptuahsation of the research into university academics' conceptions of teaching. Learning and Instruction, 7(3), 255-275.         [ Links ]

20. Kember, D., & Kwan, K. P. (2000). Lecturers' approaches to teaching and their relationship to conceptions of good teaching. Instructional Science, 28, 469-490.         [ Links ]

21. Kjellgren, K I., Hendry, G., Hultberg J., Plos, K., Rydmark, M., Tobin, G., & Säljö, R. (2008). Learning to learn and learning to teach - Introduction to studies in higher education. Medical Teacher, 30, 239-245.         [ Links ]

22. Kirton, M. J. (1991). Faking Personality Questionnaires: A Response to Furnham. Current Psychology: Research & Reviews, 10(4), 315-317.         [ Links ]

23. Lindblom-Ylänne, S., Trigwell, K., Nevgi, A., & Ashwin, P. (2006). How approaches to teaching are affected by discipline and teaching context. Studies in Higher Education, 31(3), 285-298.         [ Links ]

24. Marton, F., Hounsell, D., & Entwistle, N. J. (Eds.) (1984). The experience of learning. Edinburgh: Scottish Academic Press.         [ Links ]

25. Marton, F., & Säljö, R. (1984). Approaches to learning. In F. Marton, D. Hounsell, & N.J. Entwistle (Eds.), The experience of learning (pp. 36-55). Edinburgh: Scottish Academic Press.         [ Links ]

26. Meyer, J. H. F., & Eley, M.G. (2006). The Approaches to Teaching Inventory: A critique of its development and applicability. British Journal of Educational Psychology, 76, 633-649.         [ Links ]

27. Monroy Hernández, F. (2013). Enfoques de enseñanza y de aprendizaje de los estudiantes del Máster Universitario en Formación del Profesorado de Educación Secundaria (Doctoral Thesis, Universidad de Murcia, Spain). Retrieved from http://www.tdx.cat/bitstream/handle/10803/117259/TFMH.pdf?sequence=1.         [ Links ]

28. Montero, I., & León, O. G. (2005). Sistema de clasificación del método en los informes de investigación en Psicología. International Journal of Clinical and Health Psychology, 5(1), 115-127.         [ Links ]

29. O'Connor, B. (2000). SPSS and SAS programs for deterrrnning the number of components using parallel analysis and Velicer's MMAP test. Behavior Research Methods, Instruments & Computers, 32, 396-402.         [ Links ]

30. Oliveira Reis, L. (2009). Traducción de los cuestionarios para su uso en investigación multicultural. Actas urológicas españolas, 33 (1), 5-7.         [ Links ]

31. Pedrosa-de-Jesus, M. H., & da Silva Lopes, B. (2011). The relationship between teaching and learning conceptions, preferred teaching approaches and questioning practices. Research Papers in Education, 26 (2), 223-243.         [ Links ]

32. Postareff, L., Lindblom-Ylänne, S., & Nevgi, A. (2007). The effect of pedagogical training on teaching in higher Education. Teaching and Teacher Education, 23, 557-571.         [ Links ]

33. Prosser, M., Ramsden, P., Trigwell, K., & Martin, E. (2003). Dissonance in experience of teaching and its relation to the quality of student learning. Studies in Higher Education, 28 (1), 37-48.         [ Links ]

34. Prosser, M., & Trigwell, K. (2006). Confirmatory factor analysis of the approaches to teaching inventory. British Journal of Educational Psychology, 76, 405-419.         [ Links ]

35. Prosser, M., Trigwell, K., & Taylor, P. (1994). A phenomenographic study of academics' conceptions of science learning and teaching. Learning and Instruction, 4, 217-231.         [ Links ]

36. Richardson, J. T. E. (2004). Methodological issues in questionnaire-based research on student learning in Higher Education. Educational Psychology Review, 16 (4), 347-358.         [ Links ]

37. Rosário, P., Núñez, J. C., Ferrando, P. J., Parva, M.O., Lourenco, A., Cerezo, R., & Valle, A. (2013). Approaches to teaching and approaches to studying relationships: A two-level structural equation model for biology achievement in high school. Metacognition and Learning, 8.1 , 47-77. Doi: 10.1007/s11409-013-9095-6.         [ Links ]

38. Spector, P. (2001). Research methods in industrial and organisational psychology. In N. Anderson, D.S. Ones, H.K. Sinangil, & C. Viwesvaran (Eds.), Hand-book of industrial, work and organizational psychology (pp. 10-26). London: Sage.         [ Links ]

39. Stes, A., Coertjens, L., & Van Petegrem, P. (2010). Instructional development for teachers in higher education: impact on teaching approach. Higher Education, 60, 187-204.         [ Links ]

40. Stes, A., De Maeyer, S., & Van Petegem, P. (2010). Approaches to teaching in higher education: Validation of a Dutch version of the Approaches to Teaching Inventory. Learning Environment Research, 13, 59-73.         [ Links ]

41. Trigwell, K., & Prosser, M. (1996a). Congruence between intention and strategy in university science teachers' approaches to teaching. Higher Education, 32, 77-87.         [ Links ]

42. Trigwell, K., & Prosser, M. (1996b). Changing approaches to teaching: A relational perspective. Studies in Higher Education, 21 (3), 275-284.         [ Links ]

43. Trigwell, K., & Prosser, M. (2004). Development and use of the Approaches to Teaching Inventory. Educational Psychology Review, 16 (4), 409-424.         [ Links ]

44. Trigwell, K., Prosser, M., & Ginns, P. (2005). Phenomenographic pedagogy and a revised Approaches to Teaching Inventory. Higher Education Research & Development, 24 (4), 349-360.         [ Links ]

45. Trigwell, K., Prosser, M., & Taylor, P. (1994). Qualitative differences in approaches to teaching first year university sciences. Higher Education, 27, 75-84.         [ Links ]

46. Trigwell, K., Prosser, M. & Waterhouse (1999). Relations between teachers' approaches to teaching and students' approaches to learning. Higher Education, 37, 57-70.         [ Links ]

47. Zhang, L. F. (2001). Approaches and Thinking Styles in Teaching. The Journal of Psychology, 135 (5), 547-561.         [ Links ]

 

 

Correspondence:
Fuensanta Monroy.
Universidad de Murcia, España.
Facultad de Educación.
Dpto. Métodos de Investigación y Diagnóstico en Educación.
30100 Murcia (Spain).
E-mail: fuensanta.monroy@um.es

Article received: 13-1-2014
Revision received: 27-5-2014
Accepted: 24-6-2014