Validity and reliability of the Cyber-aggression Questionnaire for Adolescents (CYBA)

Validez y fiabilidad del Cuestionario de Ciberagresión para Adolescentes (CYBA)

 

 

David Álvarez-García, Alejandra Barreiro-Collazo, José Carlos Núñez and Alejandra Dobarro

Universidad de Oviedo, Spain

This work was financed by the Consejería de Economía y Empleo del Principado de Asturias [Council of Economy and Employment of the Princedom of Asturias] (Spain) (Ref. FC-15-GRUPIN14-053).

Correspondence

 

 

---

ABSTRACT

Cybercrime is a growing and worrisome problem, particularly when it involves minors. Cyber-aggression among adolescents in particular can result in negative legal and psychological consequences for people involved. Therefore, it is important to have instruments to detect these incidents early and understand the problem to propose effective measures for prevention and treatment. This paper aims to design a new self-report, the Cyber-Aggression Questionnaire for Adolescents (CYBA), to evaluate the extent to which the respondent conducts aggressions through a mobile phone or the internet and analyse the factorial and criterion validity and reliability of their scores in a sample of adolescents from Asturias, Spain. The CYBA was administered to 3,148 youth aged between 12 and 18 years old along with three self-reports to measure aggression at school, impulsivity, and empathy. Regarding factorial validity, the model that best represents the structure of the CYBA consists of three factors (Impersonation, Visual-sexual Cyber-aggression, and Verbal Cyber-aggression and Exclusion) and four additional indicators of Visual Cyber-aggression–Teasing/Happy Slapping. Regarding criterion validity, the score on the CYBA correlates positively with aggression at school and impulsivity and negatively with empathy. That is the way cyber-aggression correlates with these three variables, according to previous empirical evidence. The reliability of the scores on each item and factor of the CYBA are adequate. Therefore, the CYBA offers a valid and reliable measure of cyber-aggression in adolescents.

Key words: Questionnaire. Cyber-aggression. Adolescence. Evaluation.

---

RESUMEN

Los ciberdelitos constituyen un problema creciente y preocupante, sobre todo si están involucrados menores. Las ciberagresiones en adolescentes, en concreto, pueden generar consecuencias legales y psicológicas muy negativas para los implicados. Es importante, por tanto, disponer de instrumentos que permitan detectar precozmente estos hechos, así como comprender el problema de cara a plantear medidas eficaces para su prevención y tratamiento. El objetivo de este trabajo es diseñar un nuevo autoinforme, denominado Cuestionario de Ciberagresión para Adolescentes (CYBA), para evaluar en qué medida el informante ejerce agresiones a través del teléfono móvil o Internet y analizar la validez factorial y de criterio y la fiabilidad de sus puntuaciones en una muestra de adolescentes de Asturias (España). Para ello, se aplicó el CYBA a 3.148 jóvenes de 12 a 18 años, junto con tres autoinformes para medir agresión escolar offline, impulsividad y empatía. Respecto a la validez factorial, el modelo que mejor representa la estructura del CYBA es el compuesto por tres factores (suplantación, ciberagresión visual-sexual y ciberagresión verbal/exclusión) y otros cuatro indicadores de ciberagresión visual-burlas/happy slapping. Respecto a la validez de criterio, la puntuación en el CYBA correlaciona de manera positiva con agresión escolar offline e impulsividad y negativa con empatía, tres variables con las que la evidencia empírica previa indica que correlaciona la ciberagresión. La fiabilidad de las puntuaciones en cada ítem y factor del CYBA son adecuadas. Por todo ello, se concluye que el CYBA ofrece una medida válida y fiable de ciberagresión en adolescentes.

Palabras clave: Cuestionario. Ciberagresión. Adolescencia. Evaluación.

---

 

The rapid development of electronic devices and applications for communication has recently changed the way adolescents socialise. The use of such communication methods is practically universal among youth in technologically developed countries. In Spain, 90.3% of 15-year old children have a mobile phone (Instituto Nacional de Estadística–INE [National Statistics Institute], 2014), and 90% have a profile on at least one social network (Ministerio del Interior [Home Office], 2014). These percentages are similar to those from other European countries, such as France (Pouria & Brousse, 2011), Ireland (O’Neill & Dinh, 2015), or Sweden (Findahl, 2014). Despite the great advantages that these tools offer for establishing new friendships or maintaining contact with family or friends, they also present some significant potential dangers.

Sometimes adolescents take advantage of the anonymity offered by these media to intentionally harm, offend, or hurt other adolescents. The term cyber-aggression is typically used to refer to this type of behaviour (Corcoran, McGuckin, & Prentice, 2015). Aggression through electronic devices can be even more serious than face-to-face aggression. It can occur at any time of the day and can be spread immediately to many people and, because it occurs at a distance, the perpetrators do not see the negative effects of aggression on the victim, hindering empathy and promoting recurrence (Álvarez-García, Dobarro, & Núñez, 2015).

Cyber-aggression among adolescents occurs in many countries and constitutes an international problem (Li, Cross, & Smith, 2012). However, the estimated percentage of adolescents involved in such acts varies depending on the definition of cyber-aggression, the sample, and the methodology used for the analysis. A recent study in Spain based on a large nationwide sample estimates that 4.2% of students in compulsory secondary education (CSE) admit to having recorded a peer through the use of a mobile phone audio or video application to use it against him/her; 3.4% admit to having recorded a peer to later force him/her with threats to do something that he/she did not want to do; 4.6% have sent messages over the internet or by mobile phone in which a peer was insulted, threatened, offended, or frightened; and 3.8% have spread photos or images over the internet or by mobile phone of a peer to use them against him/her (Díaz-Aguado, Martínez, & Martín, 2013).

Cyber-aggression can lead to significant negative consequences in adolescent development, not only in victims but also in perpetrators. In victims, it can lead to depressive symptoms and suicidal ideation in the most severe cases (Arce, Velasco, Novo, & Fariña, 2014; Gini & Espelage, 2014). In perpetrators, their behaviour can be reinforced through peers, favouring a more general pattern of antisocial behaviour. Previous studies suggest that cyber-aggression forms part of a larger pattern of behaviour characterized by low empathy (Brewer & Kerslake, 2015), deficits of self-control (Vazsonyi, Machackova, Sevcikova, Smahel, & Cerna, 2012), and the emergence of other antisocial behaviours in the real world, including theft (Hemphill, Kotevski, & Heerde, 2015) or aggressions at school (Baldry, Farrington, & Sorrentino, 2015).

In addition to these psychological and behavioural consequences, cyber-aggression may also involve legal consequences. Many activities that define cyber-aggression constitute criminal offenses in most countries. In Spain, the Penal Code classifies the following behaviours as crimes: threats, coercion, slander, deception over the phone or internet of a minor less than 16 years old to obtain pornographic material in which a minor appears, or crimes against privacy (for example, seizing a victim's personal messages without his/her consent, accessing private or intimate data or information and sharing it, or disclosing images of another person against his/her will that were obtained with his/her consent on a personal level). According to the Organic Law 5/2000, January 12, regulating the criminal liability of minors, if a criminal act has been committed by a minor between 14 and 18 years old, corrective measures aimed at rehabilitation (such as providing community service activities, attending day care centres or institutionalisation in re-education centres) should apply instead of adult penalties (such as imprisonment or fines). Offending minors under age 14 may not be held criminally liable, and the Administration, through Social Services, is to promote measures designed to control, re-educate, and protect in cases of familial neglect. The minor's family will assume civil liability that may arise from the committed act. However, the vast majority of cyber-aggression in adolescence may have an educational solution in family and school environments before it becomes a criminal matter (Contreras, Molina, & Cano, 2011; Cutrín, Gómez-Fraguela, & Luengo, 2015).

Thus, it is important to understand this new type of aggression as well as to identify these types of incidents early to minimise their psychological and legal consequences. This process requires appropriate evaluation tools. Because cyber-aggression occurs in a virtual and often private environment, detecting it via direct observation or consultation with teachers or family members can be difficult. Even interviews or focus groups with adolescents can be fruitless because participants lack awareness of the incidents or harbour shame or fear of reporting or acknowledging particular cases of cyber-aggression, particularly those of a serious nature. Therefore, applying anonymous self-reports in which adolescents are asked to what extent they conduct or suffer from cyber-aggression may be more appropriate. Furthermore, the self-reports are more manageable as a screening measure because they can assess many people quickly; they allow coding and analysis of the obtained information more easily than other techniques, such as observation, interviews, or focus groups; and, applied collectively and anonymously, they favour honesty from the respondent.

For self-reports to be useful, their measures should be valid and reliable. In recent years, some self-reports have been published that are designed to evaluate how frequently the respondent is the perpetrator via mobile phone or the Internet and are validated with adolescents. Those validated in Spain are very rare, and the items related to cyber-aggression constitute only one factor. Both the European Cyberbullying Intervention Project Questionnaire (ECIPQ) by Del Rey et al. (2015) and the Cyberbullying Questionnaire (CBQ) by Gámez-Guadix, Villa-George, and Calvete (2014) include two factors (cyber-aggression and cyber-victimization), and the Cyberbullying test (Garaigordobil, 2013) includes three (victimization, aggression, and observation). This coincides with other validated self-reports at the international level: the Cyberbullying Scale (CS) by Menesini, Nocentini, and Calussi (2011) includes two factors (cyber-aggression and cyber-victimization); the Virtual Aggression among Students subscale by Jiménez, Castillo, and Cisternas (2012) includes two factors (“conducted or observed virtual aggressions” and “victimisation through virtual aggression”); the Multidimensional Peer Bullying Scale (MPVS-RB) by Betts, Houston, and Steer (2015) measures a factor consisting of cyber-aggression behaviours and the rest relating to offline aggression; and the cyber-aggression scale of the Revised Cyber Bullying Inventory (RCBI) by Topcu and Erdur-Baker (2010) has a unifactorial structure.

Nevertheless, from a practical point of view, it may be interesting to be able to measure different types of cyber-aggression without needing to provide the score for each of the many observable indicators (items) that these scales evaluate. Additionally, the possibility that adolescents lump together responses on items that represent different types of cyber-aggression also has theoretical interest, as it would offer empirical evidence for the definition and delimitation of the construct. None of the reviewed questionnaires that have been validated in Spain offer a multifactorial structure, but examples do exist beyond Spain. These include the E-Bullying Scale (E-BS) by Lam and Li (2013), composed of two factors (mild and serious); the cyber-aggression scale of the Cyber Victim and Bullying Scale (CVBS) by Çetin, Yaman, and Peker (2011), which includes three factors (cyber verbal bullying, hiding identity, and cyber forgery); or the Florence CyberBullying-CyberVictimization Scales (FCBVSs) by Palladino, Nocentini, and Menesini (2015), in which the cyber-aggression scale is composed of four factors (written-verbal, visual, impersonation, and exclusion). None of these three questionnaires have been validated in Spain.

Therefore, this study attempts to construct a self-report scale to measure cyber-aggression in adolescents from a multifactorial model. Among the different reviewed factorial structures, the classification based on the level of severity (Lam & Li, 2013) could be subjective and debatable because each person could experience the severity of the incident differently. As such, using the type of aggression conducted as the criterion is preferable. In this regard, the proposal by Nocentini et al. (2010) has been taken as reference, supported empirically with validation from the FCBVS scales with Italian adolescents (Palladino et al., 2015).

Therefore, this study aims to design a new self-report directed toward adolescents to evaluate the extent that the respondent conducts aggressions through a mobile phone or the internet (the Cyber-Aggression Questionnaire for Adolescents; CYBA) and to validate it in a sample of adolescents from Asturias, Spain. Two subordinate objectives summarise this overall research objective. The first is to identify the factorial structure that best represents the obtained data with the application of the questionnaire. A multifactorial model, derived from the model proposed by Nocentini et al. (2010), is expected to present a good fit to the data and adequate discriminant validity, and it would likely generate reliable scores in terms of internal consistency of the factors and representativeness of each item regarding the construct in which it is included. The second specific objective is to analyse the criterion validity of the test. The scores on the factors of the CYBA are expected to correlate positively with aggressions at school and impulsivity, and negatively with empathy, because previous research suggests that these three variables correlate this way with cyber-aggression.

 

Method

Participants

The responses of 3,148 CSE students, from 12 to 18 years old (M = 14.01, SD = 1.39), are analysed. Of the sample, 48.1% are girls. Sex is distributed homogeneously across the different ages of the sample, χ²(6) = 11.695, p = .069, Cramer's V = .061. Some 95.1% of the evaluated adolescents have their own mobile phones, 84.1% surf the internet in their free time for non-school tasks, 93.6% use instant messaging programs (Messenger, WhatsApp, or others), and 77.9% participate in social networks (Tuenti, Facebook, or others) in their free time.

The participants evaluated attended 19 schools, selected using stratified random sampling from all publicly funded schools in which CSE is taught in Asturias. To select the sample, the population of schools was divided according to their ownership (public or state-subsidised private) and, in each stratum, a number of schools proportional to the population was selected. As a result, 11 public and 8 state-subsidised private schools were selected. The selected schools are in predominantly urban areas where residents are in the middle socioeconomic class. All groups were evaluated in each selected school. A total of 28.2% of the participants are in their 1^st year, 25.3% are in their 2^nd year, 24.4% are in their 3^rd year, and 22.1% are in their 4^th year of CSE.

Measures

Cyber-Aggression Questionnaire for Adolescents (CYBA). The CYBA questionnaire designed and tested in this study is a self-report composed of 19 items, each of which describes an aggression conducted through a mobile phone or the internet (see Appendix). The evaluated participant should indicate the frequency that he/she has conducted the different actions in the last three months, using a Likert-type scale with four options (1 = never, 2 = rarely, 3 = often, and 4 = always).

 

Appendix. Indicate the frequency that you have conducted the following actions in the last three months:

1 = never, 2 = rarely, 3 = often, 4 = always.

 

Aggression at school. A self-report scale that was developed ad hoc was used; the scale was based on a selection and adaptation of six items from the Aggression Questionnaire by Álvarez-García, Menéndez, González-Castro, and Rodríguez (2012). This study uses a scale composed of the following items related to the frequency with which the participant says he/she conducts aggressions in the physical school environment: “No he dejado participar en mi grupo a algún compañero, durante alguna actividad de recreo o de educación física” [“I have not let a classmate participate in my group during the break or physical education”], “No he dejado participar en mi grupo a algún compañero en alguna actividad de clase” [“I have not let a classmate participate in my group in a class activity”], “Me he reído y burlado de algún compañero/alguna compañera” [“I have laughed at or teased a classmate”], “He hablado mal de algún compañero/alguna compañera a sus espaldas” [“I have talked badly about a classmate behind his/her back”], “He insultado a la cara a algún compañero/alguna compañera” [“I have insulted a classmate to his/her face”] and “He pegado a algún alumno/alguna alumna del centro, dentro o a la salida del recinto escolar” [“I have hit a student of the school, within or outside the school grounds”]. The response is a Likert-type list with four options (1 = never, 2 = rarely, 3 = often, 4 = always). The internal consistency of the scores obtained with this scale in this study sample is high (Raykov's rho = .917).

Impulsivity. The self-report scale is developed ad hoc from some of the criteria for impulsivity proposed by the DSM-5 for the diagnosis of ADHD (American Psychiatric Association, 2013). The scale used is composed of five items, referring to the extent that the respondent considers it true that he/she conducts certain behaviours of an impulsive nature: “En clase o en juegos, a menudo me cuesta esperar turno, por lo que me cuelo o interrumpo” [“In class or when playing games, it is often difficult for me to wait my turn, so I jump in or interrupt”], “A menudo contesto antes de que se haya completado la pregunta” [“I often answer before the question has finished”], “A menudo digo lo que me viene a la cabeza, sin pensar primero sus consecuencias o si es oportuno para la conversación” [“I often say what comes to mind without thinking first of the consequences or whether it is appropriate for the conversation”], “A menudo hago cosas sin pensar en las consecuencias” [“I often do things without thinking of the consequences”] and “Habitualmente me resulta difícil esperar turno, por lo que me adelanto a hablar cuando no me corresponde o interrumpo a quien está hablando” [“Usually, I find it difficult to wait my turn, so I jump in to speak when it is not my turn or I interrupt the person talking”]. The response is a Likert-type list with four options (1 = totally false, 2 = somewhat false, 3 = somewhat true, 4 = totally true). The internal consistency of the scores obtained with the scale in this study sample is high (Raykov's rho = .865).

Empathy. The self-report scale was developed ad hoc and composed of five items referring to the extent that the respondent considers it to be true that he/she is capable of identifying with others and sharing their feelings: “Siento las desgracias de los demás” [“I feel the misfortunes of others”], “Si se burlan de un compañero, me siento mal pensando en lo mal que lo está pasando” [“If a classmate is teased, I feel bad thinking about what is happening to him/her”], “Soy paciente con las personas que hacen las cosas peor que yo” [“I am patient with people who do things worse than I do”], “Cuando veo que un/a amigo/a está triste, yo también me entristezco” [“When I see that a friend is sad, I also become sad”] and “Me alegro cuando le pasa algo bueno a un conocido” [“I am happy when something good happens to someone I know”]. The response is a Likert-type list with four options (1 = totally false, 2 = somewhat false, 3 = somewhat true, 4 = totally true). The internal consistency of the scores obtained with the scale in this study sample is adequate (Raykov's rho = .755).

Procedure

First, the CYBA questionnaire was designed. The wording of the items came from a theoretical model of six factors or types of cyber-aggression (Table 1), based on the model of four factors (written-verbal, visual, impersonation, and exclusion) proposed by Nocentini et al. (2010). The authors produced the written items. Once the questionnaire was designed and the schools selected, permission was requested from the schools’ respective leadership teams to implement the questionnaire. Each leadership team was informed of the objectives and procedures of the study, its voluntary and anonymous nature, and the confidential treatment of the results. Once the school agreed to participate, informed consent was requested from the parents or guardians of the students, given their status as minors. The questionnaire was applied in all the schools during the second and third quarter of the 2014-2015 school year. Before answering the questionnaire, students were also informed of the anonymous, confidential, and voluntary nature of their participation. In general, students had 20 minutes to answer the questionnaires, although the timing was flexible depending on the age and characteristics of the respondents. The research team administered the test to all groups from each school during school hours.

 

Table 1. A priori Models Proposed for the Analysis of the Dimensionality of
the Cyber-Aggression Questionnaire for Adolescents (CYBA).

Note. 1FM = one-factor model, 6FM = six-factor model, 6FM2 = hierarchical model with
six first-order factors and one second-order factor.

 

Data analysis

First, in the data matrix, the presence of values that were missing or outside the scale was analysed. All items were answered in an acceptable way by more than 90% of the respondents, so none were removed from the final analysis of the test on that basis. Of the 3,233 students who responded to the CYBA, 85 responded in an acceptable way to less than 90% of the items, so they were removed from the data matrix. Once they were removed, the number of missing values in the CYBA matrix was reduced to 122 (0.2% of the total).

Second, the statistical program EQS 6.2 (Bentler, 2014) was used to analyse the normality of the data and the dimensionality of the test. Normality was assessed by analysing the multivariate kurtosis. Dimensionality was examined using confirmatory factor analysis. The multivariate kurtosis pointed to the non-normality of the obtained data (Mardia coefficient = 2571.16). Consequently, Arbitrary Generalized Least Squares (AGLS) was used as estimation method. Given the ordinal nature of the scale, the analyses were completed based on the matrix of polychoric correlations (Lila, Oliver, Catalá-Miñana, Galiana, & Gracia, 2014). Missing values were treated by computing the matrix of covariances using the pairwise method because AGLS requires large sample sizes and to avoid removing more subjects and be able to use all available data.

To determine the degree of fit of the tested models, the chi-square (χ²)/degrees of freedom (df) ratio, the Comparative Fit Index (CFI), the Bentler-Bonett Non-normed Fit Index (NNFI), the Standardized Root Mean Square Residual (SRMR), the Root Mean Square Error of Approximation (RMSEA), and the Akaike Information Criterion (AIC) were utilized. Usually, the fit is considered good when CFI ≥ .95, NNFI ≥ .95, SRMR ≤.08, and RMSEA ≤ .06 (Hu & Bentler, 1999), and χ²/df < 3 (Ruiz, Pardo, & San Martín, 2010). The AIC allows models to be compared, with those having lower values being preferable.

The discriminant validity of the model with best fit was studied, analysing the correlation between its factors. Very high correlations (r ≥ .85) warn of possible collinearity or redundancy between factors and, therefore, poor discriminant validity (Brown, 2015). Given that a very high correlation was indeed found between some factors, post hoc re-specifications in the model were completed.

The indices of fit and the correlations between factors of the two re-specified models were analysed. Once the model with the most appropriate discriminant validity and fit to the data was identified from those tested, the factor weights and standard errors of each item for that model were found. Usually, factor weights greater than .30 are considered acceptable (Izquierdo, Olea, & Abad, 2014).

Subsequently, the reliability of the scores obtained in the test was analysed. The reliability of each subscale was analysed in terms of internal consistency, finding its respective Raykov's rho from the matrix of polychoric correlations. This coefficient is considered as an alternative to the weaknesses of the Cronbach's alpha coefficient, highlighted by authors such as Sijtsma (2009). In general, the Raykov's rho coefficient is considered acceptable if it has a value above .70, high if it is greater than .80, and very high if it is greater than .90. Additionally, Squared Multiple Correlations of the scores for each item were found, which was an indicator of the proportion of variance on the item explained by the latent variable and, therefore, of the item's reliability to measure that variable (Bollen, 1989).

Finally, the statistical software SPSS 21 (IBM Corp., 2012) was used to analyse the criterion validity. For this, the Spearman correlation coefficient was calculated between the score on each of the CYBA's three factors and three external criteria about which previous evidence demonstrates their association with cyber-aggression: aggression at school, impulsivity, and empathy.

 

Results

Analysis of the Initial Theoretical Model

Factorial validity. At first, the indices of fit of the initial theoretical model, composed of significantly interrelated six factors or types of cyber-aggression (6FM), were obtained and compared with those from two additional plausible models from a theoretical perspective (Table 1). On the one hand, they were compared with a unifactorial model (1FM), whereas all the items from the test could measure only one construct (cyber-aggression). On the other hand, they were compared with a hierarchical model of six first-order factors–the same as 6FM–and one second-order factor–cyber-aggression–(6FM2), whereas each item could be explained by a first-order factor and each first-order factor by the common construct cyber-aggression. As presented in Table 2, the 6FM presents good indices of fit to the obtained data, the best from among the three models initially tested.

 

Table 2. Goodness-of-fit Indexes of the Three a priori Tested Models for the CYBA,
with the Total Sample (N = 3148).

Note. 1FM = one-factor model, 6FM = six-factor model, 6FM2 = hierarchical model with six first-order factors
and one second-order factor.χ² = chi-square, df = degrees of freedom, p = probability level, CFI = Comparative Fit Index,
NNFI = Non-Normed Fit Index, SRMR = Standardized Root Mean Square Residual,
RMSEA = Root Mean Square Error of Approximation, CI = Confidence interval, AIC = Akaike Information Criterion.

 

Discriminant validity. All the 6FM factors have positive and statistically significant correlations among themselves (Table 3). Some of these correlations are very high (r > .85): the factor Visual Cyber-Aggression Teasing/Happy Slapping is very highly correlated with Visual-Sexual Cyber-Aggression, Impersonation, and Online Exclusion; and the factor Online Exclusion is very highly correlated with Verbal-Written Cyber-Aggression and Verbal-Oral Cyber-Aggression, in addition to its relationship with Visual Cyber-Aggression Teasing/Happy Slapping referenced above.

 

Table 3. Correlation Matrix between the Components of the Six-factor Model (N = 3148).

 

Post-hoc Modifications

Due to the high correlation between some factors, two modified models of the initial 6FM were designed and tested: a model with five factors (5FM) and another with three (3FM). In both, items referring to Visual Cyber-Aggression Teasing/Happy Slapping were removed from the model and are now considered additional indicators included in the questionnaire but outside the model. Thus, the 5FM is composed of the same factors as the 6FM, with the exception of the factor Visual Cyber-Aggression Teasing/Happy Slapping. Furthermore, in the 3FM, the items initially included in the factors Verbal-Written Cyber-Aggression, Verbal-Oral Cyber-Aggression, and Online Exclusion now constitute only one factor, called Verbal Cyber-Aggression and Exclusion (Figure 1). As indicated in Table 4, both models present good indices of fit. The 5FM presents the best fit of both. Nonetheless, the correlation between the factor Online Exclusion and the two factors referring to Verbal Cyber-Aggression (Oral and Written), remain very high (r > .85). In the case of the 3FM, no correlations exist between factors greater than .85 (Figure 1). That is why this model (3FM) is considered the one that best represents the obtained data among those tested.

 

Figure 1. Factorial Structure of the CYBA. Note. I = impersonation,
V-S = visual-sexual cyber-aggression, V-E = verbal cyber-aggression and exclusion.

 

Table 4. Goodness of Fit Indexes of the Two post hoc Models, with the Total Sample (N = 3148).

Note. 3FM = three-factor model, 5FM = five-factor model.χ² = chi-square, df = degrees of freedom,
p = probability level, CFI = Comparative Fit Index, NNFI = Non-Normed Fit Index, SRMR = Standardized Root Mean Square Residual,
RMSEA = Root Mean Square Error of Approximation, CI = Confidence interval, AIC = Akaike Information Criterion.

 

Analysis of the Three-factor Model

Factor weights. In the 3FM, the factor weights of each item in its factor are high (Figure 1), with standardised regression coefficients equal to or greater than .70 on all items except 13 (λ = .69).

Reliability. The reliability of scores in each subscale, evaluated in terms of internal consistency, could be considered high, and for each item the reliability scores could be considered moderate or high (Table 5). The reliability coefficient Raykov's rho of each factor is greater than .80. The proportion of variance of the item explained by the latent variable (R²) is between 47% and 82%.

 

Table 5. Reliability of Each Item and Factor of the CYBA (N = 3148).

Note. ρ = Raykov's rho coefficient, R² = squared multiple correlation.

 

Criterion validity. The scores of the three factors of the CYBA have a statistically significant correlation with the three external criteria analysed. The correlation is positive with the frequency of aggression conducted in the physical school environment and impulsive behaviours, reported by the evaluated adolescents themselves; the correlation is negative with empathy (Table 6).

 

Table 6. Spearman's Rank Correlation Coefficients between the Score on Each Factor
of the CYBA and the Scores on the Aggression at School, Impulsivity, and Empathy Scales (N = 3148).

* - p < .01.

 

Discussion

This study aimed to design a new self-report directed toward adolescents to evaluate the extent that the respondent conducts aggressions through a mobile phone or the internet (the Cyber-Aggression Questionnaire for Adolescents - CYBA) and to validate it in a sample of adolescents from Asturias (Spain). The results obtained indicate that the CYBA presents adequate statistical assurance for its use for its designed purpose.

The first specific objective of this study was to identify the factorial structure that best represents the data obtained through implementation of the questionnaire. As was initially hypothesised, the model that best fits the data is multifactorial. However, the initially proposed six-factor model presented a very high correlation between the scores of some of its factors. The tested model, which best represents the obtained data with adequate discriminant validity, is composed of three factors (Impersonation, Visual-Sexual Cybervictimization, and Verbal Cybervictimization and Exclusion), in which the four items of Visual Cybervictimization Teasing/Happy Slapping are extracted from the model and remain complementary indicators. The previously validated questionnaires in Spain had factor structures in which cyber-aggression constituted only one factor (Del Rey et al., 2015; Gámez-Guadix et al., 2014; Garaigordobil, 2013). Therefore, an overall score on cyber-aggression could be obtained using these, as well as a specific score on each indicator (item) separately. However, the results obtained in this study offer empirical support for the idea that the cyber-aggression construct consists of significantly interrelated but distinct types, and they justify the collection of scores for each of those types as well as for each observable indicator (item). Other questionnaires published in the international sphere also involve multifactorial structures (CVBS, Çetin et al., 2011; E-BS, Lam & Li, 2013; FCBVSs, Palladino et al., 2015). However, the CYBA differs from them in the types of cyber-aggression that it evaluates and in the observable indicators that define them. Similar to the CYBA, these three questionnaires include items related to verbal cyber-aggression and two of them also include items related to impersonation. However, only the CYBA offers a specific score on visual cyber-aggression of a sexual nature, differentiated from other indicators of visual cyber-aggression linked to teasing and physical aggression. Furthermore, only the CYBA and the scale on cyber-aggression by Palladino et al. (2015) include items related to online exclusion. The results obtained in this study, unlike those obtained by Palladino et al. (2015), do not justify the consideration of online exclusion as a type of cyber-aggression distinct from verbal cyber-aggression.

The reliability analyses support the relevance of the three-factor model considered later. The indices of internal consistency of the scores obtained for each factor or type of cyber-aggression suggest that the items or observable indicators that shape them measure the same latent variable without being redundant. Furthermore, the squared multiple correlation of each item demonstrates that all indicators utilized are relevant for measuring the type of cyber-aggression included.

The second specific objective of this study has been to analyse the criterion validity of the test. As had been hypothesised, the scores obtained with the CYBA have a statistically significant correlation with three external variables, which, according to previous evidence, correlate with cyber-aggression in adolescence. The score on the CYBA correlates positively with the frequency with which the adolescent reports having attacked his/her classmates at their school and with the frequency with which he/she admits conducting impulsive behaviours, and it correlates negatively with empathy. These results are consistent with those presented in the recent review paper by Baldry et al. (2015), which, similar to this study, concludes that the variable most associated with cyber-aggression in adolescence is being a perpetrator in the physical school context.

This study has various theoretical and practical implications. From the theoretical point of view, it provides empirical evidence that contributes to the conceptual delimitation of cyber-aggression in adolescence, the distinction of types, and the identification of representative observable indicators of each. The results obtained support the hypothesis of the multifactorial nature of cyber-aggression in adolescence and the suitability of the observable indicators included in the CYBA. From a practical standpoint, it offers professionals in the field of psychology a tool to evaluate cyber-aggression that is concise, simple to implement, code, and analyse, economical in terms of time and cost, compared to other evaluation methods, and with adequate metric properties. In the school context, CYBA offers a screening measure to detect cyber-aggression among adolescents. Early identification, accompanied by educational treatment, can prevent it from becoming a criminal problem. In the context of juvenile justice, the CYBA helps analyse the evolution of cyber-aggression (if it persists or stops in adulthood, if it evolves into another type of criminal activity) and its major risk factors to design preventive interventions, both at the school and community levels, and to guide the treatment of perpetrated cases. The relationship observed in this study and in previous studies (Baldry et al., 2015) between cyber-aggression and the three analysed external criteria suggests that including the development of empathy and behavioural self-control as well as the prevention of aggression among students in the educational context benefits the prevention and treatment of cyber-aggression in adolescence.

Therefore, this study proposes a contribution to the study of cyber-aggression in adolescence. However, it also has some limitations, three of which must be highlighted. First, because the CYBA is a self-reporting tool, the results could be affected by response biases, such as falsification or social desirability (Jiménez, Sánchez, & Tobón, 2009). In the future, CYBA could include a scale for truthfulness. Second, aggression at school, impulsivity, and empathy have been evaluated using self reports developed ad hoc. Although its metric properties have been adequate in the sample analysed in this study, it would have been preferable to use measures whose scores had already been validated in the Spanish adolescent population (Torregrosa, Inglés, Estévez-López, Musitu, & García-Fernández, 2011). Finally, the CYBA has been tested with a large, randomly selected sample, but it was taken from a particular population and limited to certain ages and geographical areas. Therefore, any generalisation of this study's findings to other ages or regions should be done with caution (Muñiz, Elosua, & Hambleton, 2013). Looking ahead, it would be of interest to validate the test in other age groups and cultural contexts.

 

Conflict of Interest

The authors of this article declare no conflict of interest.

 

References

1. Álvarez-García D., Dobarro A., Núñez J.C. Validez y fiabilidad del Cuestionario de cibervictimización en estudiantes de Secundaria (Validity and reliability of the Cybervictimization questionnaire in secondary education students). Aula Abierta. 2015;43(1):32-38. http://dx.doi.org/10.1016/j.aula.2014.11.001.         [ Links ]

2. Álvarez-García D., Menéndez S., González-Castro P., Rodríguez C. Hiperactividad-impulsividad y déficit de atención como predictores de participación en situaciones de violencia escolar (Hyperactivity-impulsivity and attention deficit as predictors of involvement in school violence situations). International Journal of Psychology and Psychological Therapy. 2012;12:185-202.         [ Links ]

3. American Psychiatric Association, 2013. Diagnostic and Statistical Manual of Mental Disorders (DSM-5). Washington. DC: American Psychiatric Publishing, 2013.         [ Links ]

4. Arce R., Velasco J., Novo M., Fariña F. Elaboración y validación de una escala para la evaluación del acoso escolar (Development and validation of a scale to assess bullying). Revista Iberoamericana de Psicología y Salud. 2014;5(1):71-104.         [ Links ]

5. Baldry A.C., Farrington D.P., Sorrentino A. Am I at risk of cyberbullying? A narrative review and conceptual framework for research on risk of cyberbullying and cybervictimization: The risk and needs assessment approach. Aggression and Violent Behavior. 2015;23:36-51. http://dx.doi.org/10.1016/j.avb.2015.05.014.         [ Links ]

6. Bentler P.M. EQS 6. 2 for Windows (Build 107) - Structural Equation Modeling Software. Encino, CA: Multivariate Software, Inc., 2014.         [ Links ]

7. Betts L.R., Houston J.E., Steer O.L. Development of the Multidimensional Peer Victimization Scale-Revised (MPVS-R) and the Multidimensional Peer Bullying Scale (MPVS-RB). The Journal of Genetic Psychology. 2015;176:93-109. http://dx.doi.org/10.1080/00221325.2015.1007915.         [ Links ]

8. Bollen K.A. Structural Equations with Latent Variables. New York: John Wiley & Sons, Inc., 1989.         [ Links ]

9. Brewer G., Kerslake J. Cyberbullying, self-esteem, empathy and loneliness. Computers in Human Behavior. 2015;48:255-260. http://dx.doi.org/10.1016/j.chb.2015.01.073.         [ Links ]

10. Brown T.A. Confirmatory Factor Analysis for Applied Research. (2^nd Edition), New York: The Guilford Press, 2015.         [ Links ]

11. Çetin B., Yaman E., Peker A. Cyber victim and bullying scale: A study of validity and reliability. Computers & Education. 2011;57:2261-2271. http://dx.doi.org/10.1016/j.compedu.2011.06.014.         [ Links ]

12. Contreras L., Molina V., Cano M.C. In search of psychosocial variables linked to the recidivism in young offenders. The European Journal of Psychology Applied to Legal Context. 2011;3:77-88.         [ Links ]

13. Corcoran L., Mc C., Prentice G. Cyberbullying or Cyber Aggression? A Review of Existing Definitions of Cyber-Based Peer-to-Peer Aggression. Societies. 2015;5:245-255. http://dx.doi.org/10.3390/soc5020245.         [ Links ]

14. Cutrín O., Gómez-Fraguela J.A., Luengo M.A. Peer-group mediation in the relationship between family and juvenile antisocial behavior. The European Journal of Psychology Applied to Legal Context. 2015;7:59-65. http://dx.doi.org/10.1016/j.ejpal.2014.11.005.         [ Links ]

15. Del Rey, R., Casas, J. A., Ortega-Ruiz, R., Schultze-Krumbholz, A., Scheithauer, H., Smith, P., ...Plichta, P. (2015). Structural validation and cross-cultural robustness of the European Cyberbullying Intervention Project Questionnaire. Computers in Human Behavior, 50, 141-147. doi: 10.1016/j.chb.2015.03.065.         [ Links ]

16. Díaz-Aguado M.J., Martínez R., Martín J. El acoso entre adolescentes en España. Prevalencia, papeles adoptados por todo el grupo y características a las que atribuyen la victimización (Bullying among adolescents in Spain. Prevalence, participants¿ roles and characteristics attributable to victimization by victims and aggressors). Revista de Educación. 2013;362:348-379. http://dx.doi.org/10.4438/1988-592X-RE-2011-362-164.         [ Links ]

17. Findahl, O. (2014). Svenskarna och internet 2014 (The Swedes and the internet 2014). Stockholm: Stiftelsen för internetinfrastruktur. Retrieved from https://www.iis.se/docs/SOI2014.pdf.         [ Links ]

18. Gámez-Guadix M., Villa-George F., Calvete E. Psychometric properties of the Cyberbullying Questionnaire (CBQ) among Mexican adolescents. Violence and Victims. 2014;29:232-247. http://dx.doi.org/10.1891/0886-6708.VV-D-12-00163R1.         [ Links ]

19. Garaigordobil, M. (2013). Cyberbullying. Screening de acoso entre iguales (Cyberbullying: Screening of peer harassment). Madrid: TEA Ediciones.         [ Links ]

20. Gini G., Espelage D.L. Peer Victimization, Cyberbullying, and Suicide Risk in Children and Adolescents. JAMA. 2014;312:545-456. http://dx.doi.org/10.1001/jama.2014.3212.         [ Links ]

21. Hemphill S.A., Kotevski A., Heerde J.A. Longitudinal associations between cyber-bullying perpetration and victimization and problem behavior and mental health problems in young Australians. International Journal of Public Health. 2015;60:227-237. http://dx.doi.org/10.1007/s00038-014-0644-9.         [ Links ]

22. Hu L., Bentler P.M. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal. 1999;6:1-55. http://dx.doi.org/10.1080/10705519909540118.         [ Links ]

23. IBM, 2012. IBM Corp. (2012). SPSS Statistics for Windows, Version 21.0. Armonk, NY: IBM Corp.         [ Links ]

24. Instituto Nacional de Estadística (2014). Encuesta sobre Equipamiento y Uso de Tecnologías de Información y Comunicación en los hogares 2014 (Survey on the Equipment and Use of Information and Communication Technologies in Households. Year 2014). Retrived from http://www.ine.es/jaxi/menu.do?type=pcaxis&path=/t25/p450/base_2011/a2014/&file=pcaxis.         [ Links ]

25. Izquierdo I., Olea J., Abad F.J. Exploratory factor analysis in validation studies: Uses and recommendations. Psicothema. 2014;26:395-400. http://dx.doi.org/10.7334/psicothema2013.349.         [ Links ]

26. Jiménez A.E., Castillo V.D., Cisternas L.C. Validación de la escala de agresión entre pares, y subescala de agresión virtual en escolares chilenos (Validation of the Aggression Among Peers Scale and Virtual Aggression Sub-scale with Chilean students). Revista Latinoamericana de Ciencias Sociales, Niñez y Juventud. 2012;10:825-40.         [ Links ]

27. Jiménez F., Sánchez G., Tobón C. A social desirability scale for the MMPI-2. Which of the two: Wiggins (WSD) or Edwards (ESD)?. The European Journal of Psychology Applied to Legal Context. 2009;1:147-163.         [ Links ]

28. Lam L.T., Li Y. The validation of the E-Victimisation Scale (E-VS) and the E-Bullying Scale (E-BS) for adolescents. Computers in Human Behavior. 2013;29:3-7. http://dx.doi.org/10.1016/j.chb.2012.06.021.         [ Links ]

29. Li Q., Cross D., Smith P.K. Cyberbullying in the Global Playground: Research from International Perspectives. Chichester: Wiley-Blackwell, 2012.         [ Links ]

30. Lila M., Oliver A., Catalá-Miñana A., Galiana L., Gracia E. The Intimate Partner Violence Responsibility Attribution Scale (IPVRAS). The European Journal of Psychology Applied to Legal Context. 2014;6:29-36. http://dx.doi.org/10.5093/ejpalc2014a4.         [ Links ]

31. Menesini E., Nocentini A., Calussi P. The measurement of cyberbullying: dimensional structure and relative item severity and discrimination. Cyberpsychology, Behavior, and Social Networking. 2011;14:267-274. http://dx.doi.org/10.1089/cyber.2010.0002.         [ Links ]

32. Ministerio del Interior (2014). Encuesta sobre hábitos de uso y seguridad de Internet de menores y jóvenes en España (Survey on Internet use and safety of children and youth in Spain). Retrived from http://www.interior.gob.es/documents/10180/2563633/Encuesta+sobre+h%C3%A1bitos+de+uso+y+seguridad+de+internet+de+menores+y+j%C3%B3venes+en+Espa%C3%B1a/b88a590a-514d-49a2-9162-f58b7e2cb354        [ Links ]

33. Muñiz J., Elosua P., Hambleton R.K. International Test Commission Guidelines for test translation and adaptation: Second edition. Psicothema. 2013;25:151-157. http://dx.doi.org/10.7334/psicothema2013.24.         [ Links ]

34. Nocentini A., Calmaestra J., Schultze-Krumbholz A., Scheithauer H., Ortega R., Menesini E. Cyberbullying: Labels, behaviours and definition in three European countries. Australian Journal of Guidance & Counselling. 2010;20:129-142.         [ Links ]

35. O´Neill B., Dinh T. Net Children Go Mobile: Full findings from Ireland. Dublin: Dublin Institute of Technology, 2015.         [ Links ]

36. Palladino B.E., Nocentini A., Menesini E. Psychometric Properties of the Florence CyberBullying-CyberVictimization Scales. Cyberpsychology, Behavior, and Social Networking. 2015;18:112-119. http://dx.doi.org/10.1089/cyber.2014.0366.         [ Links ]

37. Pouria, S., & Brousse, M. (2011). Enfants et Internet. Barométre 2011 del´ opération nationale de sensibilisation: Génération numérique (Children and Internet. Barometer 2011 of the national sensitization campaign: Digital Generation). Montreuil: CALYSTO & La Voix de L´enfant. Retrieved from http://www.generationnumerique.com/wp-content/uploads/2012/02/Barometre-Calysto-F%C3%A9vrier-2012-BD.pdf.         [ Links ]

38. Ruiz M.A., Pardo A., San R. Modelos de ecuaciones estructurales (Structural Equation Models). Papeles del Psicólogo. 2010;31:34-45.         [ Links ]

39. Sijtsma K. On the use, misuse, and the very limited usefulness of Cronbach's Alpha. Psychometrika. 2009;74:107-120. http://dx.doi.org/10.1007/s11336-008-9101-0.         [ Links ]

40. Topcu Ç., Erdur-Baker Ö. The Revised Cyber Bullying Inventory (RCBI): validity and reliability studies. Procedia Social and Behavioral Sciences. 2010;5:660-664. http://dx.doi.org/10.1016/j.sbspro.2010.07.161.         [ Links ]

41. Torregrosa M.S., Inglés C.J., Estévez-López E., Musitu G., García-Fernández J.M. Evaluación de la conducta violenta en la adolescencia: Revisión de cuestionarios, inventarios y escalas en población española (Assessment of violent behaviour in adolescence: Review of questionnaires, inventories, and scales in Spanish population). Aula Abierta. 2011;39:37-50.         [ Links ]

42. Vazsonyi A.T., Machackova H., Sevcikova A., Smahel D., Cerna A. Cyberbullying in context: Direct and indirect effects by low self-control across 25 European countries. European Journal of Developmental Psychology. 2012;9:210-227. http://dx.doi.org/10.1080/17405629.2011.644919.         [ Links ]

 

 

Dirección para correspondencia:
David Álvarez-García
Facultad de Psicología.
Universidad de Oviedo.
Plaza Feijóo, s/n.
33003 Oviedo, Spain.
alvarezgardavid@uniovi.es

Received 26 October 2015
Received in revised form 11 February 2016
Accepted 11 February 2016