Servicios personalizados
Inglés (pdf)
Articulo en XML
Referencias del artículo
Traducción automática
Enviar articulo por email
Citado por SciELO 
Citado por Google 
Similares en
SciELO 
Similares en Google 
Permalink
Adolescence is a developmental period in which people are most vulnerable to multiple overlapping health risk behaviors, including unprotected sex, driving-recklessly, drug use and interpersonal violence (Fernández-Hermida & Villamarín-Fernández, 2021; Weidberg et al., 2022). Substance use and other addictive behaviors, particularly excessive alcohol use, are one of the main causes of loss in quality of life and morbi-mortality in childhood and adolescence (Frobel et al., 2022; Wilson & Fergusson, 2022). Between 38.2% and 73.9% of 14-18-year-old Spanish students have used alcohol and tobacco in their lifetimes, while rates of regular (at least monthly) use are 53.6% for alcohol and 23.9% for tobacco (Observatorio Español de las Drogas y las Adicciones [OEDA], 2021a). Cannabis is the most widely-used illicit drug and concerns have been raised about its increasing use in the young population. Gambling (land-based and online) has shown an upward trend in recent years, and it is estimated that 25.5% of 14-18-year-old Spanish students have gambled either online or land-based. Gaming is also a widespread behavior and over 20% of 14-18 year-olds engage in problematic online gaming (OEDA, 2021b).
Numerous individual, family, and societal factors place adolescents at particular risk of using drugs and experiencing negative consequences (Fernández-Calderón et al., 2022; Gray & Squeglia, 2018). Sensation-seeking (Aguilar et al., 2022; Fernández-Artamendi et al., 2016), impulsivity (Eslava et al., 2022; Martínez-Loredo et al., 2015), social conformity (Santor et al., 2000), emotional dysregulation (Momeñe et al., 2021), and parenting styles (Al-Halabí et al., 2006; Martínez-Loredo et al., 2016) are amongst the most important factors in predicting early substance use. Other research suggests psychological disorders or related problems (Bousoño et al., 2021; Fernández-Artamendi et al., 2021) and poor school adjustment increase the risks of substance use problems (Bradshaw et al., 2014; Carbonneau et al., 2022).
In recent decades, increasing efforts have been made to prevent substance use in the adolescent population, with a plethora of interventions being developed (Foxcroft & Tsertsvadze, 2011; Maalouf et al., 2019; MacArthur et al., 2018). Universal prevention in the school setting is considered one of the most suitable, cost-effective interventions for reducing first use and preventing the transition from experimental to problematic patterns of substance use (Faggiano et al., 2014). Rooted in the developmental framework, these interventions share their focus on strengthening individuals' resources that help young people to be able to accomplish their goals through life-skills training, emotional education, and self-control (problem solving) (Burkhart et al., 2022). Furthermore, prevention curricula usually cover other specific substance-use variables, such as normative beliefs, perceptions of harm, and decision-making (refusal) skills (Komro, 2020). Unfortunately, there are limited assessments on long-term effectiveness of prevention programs in Spain and the core mechanisms of change (mediators and moderators) are rarely explored (Espada et al., 2015).
Research in the field of addiction has recently shown interest in understanding it from a transdiagnostic perspective, exploring variables that can account for the onset, progression and maintenance of different substance and non-substance use behaviors and other co-occurring conditions (Dalgleish et al., 2020; Kim et al., 2020). This relatively new approach has produced a significant body of research that supports, amongst other things, the importance of emotion regulation and negative urgency in predicting addictive behaviors (Weiss et al., 2022; Wolitzky-Taylor et al., 2016). Accordingly, there has been some action to promote transdiagnostic interventions in the addiction field, but there are only a handful of studies, primarily in adults (Martínez-Loredo & Fernández-Hermida, 2019).
The “Juego de Llaves” [Spanish for “Set of Keys”] program (Asociación Proyecto Hombre, 2014) is a manual-based universal school prevention program that has been adapted from the ‘Between All' program (Gil Carmena, 2005). The program was strongly influenced by ‘Construyendo Salud', the Spanish adaptation of the Life Skills Training (an evidence-based intervention for substance use to be delivered in primary education and compulsory secondary education) (Botvin & Griffin, 2002; Gómez-Fraguela et al., 2002; 2003). The main goal of this program is to prevent substance use and other addictive behaviors. It is to be delivered by teachers in a six-to-twelve-session format through different modules that cover emotional regulation, cognitive strategies, social interaction, free time, education on drugs, and life values (Asociación Proyecto Hombre, 2014). “Juego de Llaves” is being implemented in over 32 schools in Spain but its efficacy has yet to be evaluated.
Given this background, the present study aims to describe the design of an independent external evaluation (outcome and process) for the “Juego de Llaves” prevention program. Specifically, it reports on the following areas: i) the randomization procedure to evaluate efficacy; ii) follow-up schedule; iii) data collection procedures, and iv) outcome variables. This will be amongst the few long-term evaluations of a school-based prevention program in a natural setting, involving schools that are already implementing the program in Spain and including a broad transdiagnostic assessment protocol that seeks to identify potential mechanisms of behavioral change. There is a pressing international need to scale up effective programs, and there have been calls for evaluations that identify which prevention programs work for whom, for how long, and in which conditions (United Nations Office on Drugs and Crime [UNODC], 2018). This study will make a substantial contribution to this endeavor and serve as a source of inspiration for those working in prevention in the addiction field.
Method
Participants
Participating schools will be selected randomly from the schools that are already implementing the “Juego de Llaves” program in Spain, which includes 32 schools in almost all of the regions (i.e., autonomous communities) in Spain. Participants will be adolescents aged 12-16 in compulsory secondary education (ESO). All students, including those with educational needs, will receive the intervention. The final sample size will depend on the specific statistical approach adopted: descriptive statistics (prevalence) and independent t-tests (mean differences). For the first approach, considering a statistical power of 80% and a 95% CI, a minimum of 1,085 participants would be needed to detect small effect sizes ((0.2). For the second approach, a minimum of 2,096 participants are needed. With a power of 80% (95% CI), this sample size would ensure the detection of small effect sizes ((0.2). Therefore, considering an expected intra-group dependency of .02, the minimum size is estimated at 2,096 participants.
Students will be included in this study provided they and their parents give their informed consent (by written statements) to participate. Only participants completing the baseline assessment will be included in the efficacy analyses. Incomplete or inadequate completion (e.g., acquiescence, random response) of baseline e-assessments will be considered exclusion criteria.
Instruments
An overview of the assessment measures is provided in Table 1. A wide range of tools are included to assess sociodemographic, family, and contextual variables. Most of the drug use assessment measures are validated in the Spanish-speaking adolescent samples, with the exception of the following: the Child and Youth Resilience Measure (CYMR; Resilience Research Centre, 2018), the Monetary Choice Questionnaire (MCQ; Kirby et al., 1999), the Adolescent Reinforcement Survey Schedule (ARSS-SUV; adapted from Murphy et al., 2005), and the Substance and Activity use Problem Index (SAUPI), which was designed by the research team specifically for this project given no other similar or more appropriate tool was available at the time the protocol was designed. Of note is that the Brief Alcohol Density Assessment (BASDA; Fortune et al., 2013) was adapted to adolescents (the Brief Drug Use Density Assessment, [BDUDA]) by the research team and with the purpose of measuring other substances than alcohol, as it was originally designed.
The instruments cover each of the program areas such that there are no areas without an instrument or instruments that do not correspond to one of the areas addressed by Juego de Llaves. Psychological and transdiagnostic variables are also considered. Five attentional control items will be included in the assessment battery at random (e.g., for this question choose “true”). To avoid bias, response options for attentional checks will follow the response options scales at next preceding.
Table 1. Overview of the assessment measures at pre, post and follow-ups.
| Area | Measure | Reliability |
|---|---|---|
| Demographics | Region, age, course, sex, born in Spain (yes/no) | NA |
| Lifestyle | Hours of sleep, time to fall asleep Frequency of free-time activities (items from the Spanish epidemiological survey; OEDA, 2021a) | NA |
| Socioeconomic level | FAS-II (Boyce et al., 2006) | α = .31-.47 (Pérez et al., 2021) |
| Academic performance | Academic performance, failed subjects, truancy, repeated school year, sense of belonging at school | NA |
| Psychological variables Resilience Behavioral problems Emotion reg valign="top"ulation Suicidal behavior Impulsivity | CYMR (Resilience Research Centre, 2018) SDQ (Ortuño-Sierra et al., 2018) ERQ-CA (Navarro et al., 2018) PSS (Fonseca-Pedrero et al., 2018) BUPPS-P-NA (Caneto et al., 2020) MCQ (Kirby et al., 1999) DD (García-Rodríguez et al., 2013) | Not yet examined in Spanish-speaking adolescent samples Ordinal α = .88 α = .61-.64 Ordinal α = .93 ω = .64-.83 Not yet examined in Spanish-speaking adolescent samples. (α= 0.90) (Martínez-Loredo et al., 2017) |
| Addictive behaviors Attitudes Substance use Gambling and gaming Con valign="top"sequences of addictive behavior Substance/non-substance reinforcement ratio | Attitudes towards drug use Tobacco, cannabis, alcohol (ever, past year/month use) Past year and past month Ad-hoc questionnaire (Substance and Activity use Problem Index [SAUPI]; unpublished) ARSS-SUV (adapted from Murphy et al., 2005) | NA NA NA Not yet examined in Spanish-speaking adolescent samples Not yet examined in Spanish-speaking adolescent samples |
| Substance-related social network | Brief Drug Use Density Assessment [BDUDA; adapted from Fortune et al., 2013) | Not yet examined in Spanish-speaking adolescent samples |
Note. NA = Not applicable; FAS = The Family Affluence Scale; CYMR = The Child and Youth Resilience Measure; SDQ = Strengths and Difficulties Questionnaire (25-item version); ERQ-CA = The Emotional Regulation Questionnaire - adolescents; PSS = The Paykel Suicide Scale; BUPPS-P-NA= The Scale of impulsivity for children and adolescents; MCQ= The Monetary Choice Questionnaire (21-item version); DD = Delay discounting; SAUPI = Substance and Activity Use Problem Index; ARSS-SUV = The Adolescent Reinforcement Survey Schedule; BDUDA = The Brief Drug Use Social Density Assessment.
Procedure
The Juego de Llaves program
Juego de Llaves is a universal prevention program aimed at reducing addictive behaviors, including drugs of abuse and behavioral addictions (gambling and gaming). The program is targeted at students in compulsory secondary education and is intended to train individuals in effective skills for coping with high-risk situations for drug use, and develop prosocial attitudes and a healthy life-style. Curriculum materials have been developed to standardize the program and ensure the fidelity of the implementation. A manual for teachers where modules and objectives are described has been specifically developed.
Juego de Llaves addresses specific vulnerability factors for drug use, such as sexual and affective relationships and peer violence. It contains six different modules (see Table 2 for an overview) and teachers are encouraged to deliver at least one activity per module during four academic course years. The program is framed on the biopsychosocial model (Engel, 1980) which poses that a set of macro- and micro-situational factors interact with individual variables to account for addictive behaviors. As in the original program by Botvin (Botvin & Griffin, 2001), the theoretical background of Set of Keys includes key elements from social learning theory (Bandura, 1977), meaning engagement in drug use or engagement in other addictive behaviors depends on environmental (e.g., accessibility) and individual (e.g., personality) factors, but also on peers' influence, parenting styles and other social influences (e.g., social norms).
Table 2. Overview of the Juego de Llaves prevention program.
| Module | Objectives |
|---|---|
| Emotional management | - Recognize and manage individuals' own emotions - Improving emotional management in high-risk situations for drug use or engaging in other addictive behaviors |
| Cognitive strategies | - Training in cognitive strategies associated with decision making, critical thinking and attributional styles |
| Social interaction | - Recognize the relevance of social relationships - Training in effective social communication interaction skills |
| Leisure and free time activities | - Psychoeducation on leisure and free time: activities and function - Discussion of risk and beneficial aspects of strategies for a healthy use of information and communication technologies |
| Information on drug use and other addictive behaviors | - Drug use information and other addictive behaviors - Promoting critical thinking on myths around drug use and other addictive behaviors |
| Values | - Identify relevant values in the adolescence - Deliberate on individuals' own values |
Design of the evaluation
Following gold standards in research on intervention effectiveness, the study design including the proximal and distal outcomes has been preregistered in ClinicalTrials.gov (ref: NCT05442294) (Hariton & Locascio, 2018). During academic years 2022-2025, the “Juego de Llaves” universal school prevention program will be evaluated in a set of schools in Spain. Following a logit model, outcome evaluation will be through six different stages (see Figure 1). The first stage will consist of developing an e-assessment (app) comprising the questionnaires and items for the outcome evaluation. Despite the use of e-assessments in prevention being an uncommon practice, these procedures are deemed feasible and reliable in school settings (Martínez-Loredo et al., 2021) and are though to improve the measurement of sensitive information such as drug use (Bryan et al., 2022). E-assessments have multiple benefits over pen-and-paper procedures, including cost-effectiveness, the possibility of implementing behavioral tasks with low face validity, and enhanced internal validity and quality control through branches and algorithms that give warning prompts on mandatory questions (Balaskas et al., 2021; Burgess-Hull & Epstein, 2021; Fonseca-Pedrero et al., 2022).
Before initiating the pilot study, the researchers will deliver training to personnel in charge of the evaluation (professionals from outside the school setting with experience in the prevention field) focused on technical procedures (how to use the tablets the app is installed on) and quality control (e.g., ensuring students answer the questions independently).
The study protocol will be sent for review to the Ethics Committee of the University of Oviedo and informed consent will be collected from parents/guardians and students (stage 2). The pilot of the assessment battery (stage 3) will be conducted with students enrolled in the four years of compulsory secondary education in Spain. The rationale is to include the widest range of potential responses and to check the suitability of both the procedure and items for the target population. This pilot study will allow us to readjust the assessment battery and estimate the sample size for assessing effectiveness (stage 4). The pilot testing of the e-assessment will take place in two schools that are already applying “Juego de Llaves”. A sample size of at least 250 participants will be considered on the basis that one participant per item test, with a minimum of 100, is deemed adequate according to existing guidelines (Anthoine et al., 2014). Descriptive data from e-assessment (means, frequencies) will be examined and magnitudes of associations will be used to conduct a priori estimations of the sample size in the efficacy assessments. Stage 5 will consist of matching control schools to intervention groups, balancing for community and participant (student) characteristics. This means that schools will work as ‘blocks' in a Non-Randomized Control Cluster Trial. This design is appropriate for naturalistic research, where it is not feasible to randomly assign individuals to clusters. The intervention condition will receive the “Juego de Llaves” program and the control arm will be non-active (non-structured prevention initiatives as usual).
The evaluation of “Juego de Llaves” will look at both process and outcome evaluations (stage 6). The process evaluation will follow best practice guidelines (Baranowski & Stables, 2000; Chacón Moscoso et al., 2013) and include the following outputs: i) recruitment and retention of participants, ii) degree of program implementation (fidelity to the program manual), and iii) barriers to implementation. For this outcome, focus groups with the intervention implementers (teachers) are planned, combining quantitative and qualitative (interviews, content analysis) approaches. This step is necessary when it comes to evaluating the feasibility and suitability of the program and identifying areas to be improved, amended, or removed (Medeiros et al., 2016).
The outcome evaluation (pre-post changes) will look at the short- and long-term effectiveness (12-, 24- and 36-months) of the program using the following variables: past-month substance use, family, and psychological variables. Primary outcomes will be as follows:
- Distal variables. Substance use (ever used, past-year and past-month tobacco, alcohol, and cannabis use), non-substance use behaviors (past-year and past-month gambling and gaming) and consequences of use. Past-month drunkenness episodes will be considered as well given the high prevalence of alcohol use in Spain.
- Proximal variables. These will involve the variables targeted in the “Juego de Llaves” program: emotional regulation, cognitive strategies, social interaction, leisure and free time, life values, drug use information, and attitudes towards drug use.
- Moderators. Sociodemographic variables (sex, age, academic performance) and psychological variables other than the proximal variables will be tested as variables affecting the observed changes.
Data analysis
A pipeline statistical approach is planned as follows. First, the sample size for the efficacy and effectiveness analyses will be estimated considering intraclass correlations (ICC) between participants' measures in the participating schools. ICC quantifies the degree of similarity of the clusters (data at the school level) and indicates how substance use rates vary at cluster level (Eldridge et al., 2009). Research has shown that even a small positive ICC can produce large effects in the variance and test hypothesis estimations, leading to underestimating standard errors and a high chance of Type I errors (MacKinnon & Lockwood, 2003). This is the case in many school-based prevention programs that have shown small ICCs of effects for common drug use measures historically (ICC ranging from r = .02-.05) (Murray & Hannan, 1990). This is particularly problematic as the lower the ICC, the more likely there are to be significant effects (Shackleton et al., 2016).
Second, descriptive analyses will be conducted to report on sample characteristics and feasibility of data collection more broadly. Following the recommendations from the International Test Commission, psychometric properties of the tests will also be examined (Hernández et al., 2020). Analyses will be performed using both Classical Test and Item Response Theories (Muñiz, 2010). Item analyses (mean and standard deviations, skewness, kurtosis, item discrimination) will be performed. Reliability and validity of the ad-hoc measures will be estimated. The model for the evaluation of tests by the Spanish Test Commission will be used as a guide to analyze the quality of the assessment based on its characteristics (CET-R; Hernández et al., 2016; Hernández et al., 2022).
Analysis of the outcome evaluation (effectiveness assessments) will be conducted using an intention-to-treat approach, meaning all participants successfully completing the baseline and receiving the first intervention session will be included in subsequent analyses.
Discussion
Following best practices in prevention science (European Monitoring Centre for Drugs and Drug Addiction [EMCDDA], 2010; Galanti, 2021; Gottfredson et al., 2015), we reported on the design of the evaluation of “Juego de Llaves”, a manualized universal prevention program targeted at substance use and other addictive behaviors in adolescents. The paper contributes to the open science movement by providing transparency of the evaluation procedure, program implementation and reproducibility (Grant et al., 2022). Results from this non-randomized control cluster trial will inform stakeholders about the sustainability of the program by providing data on long-term effectiveness in a large sample of adolescents in Spain.
In Spain, the Best Practice Portal (a recognized e-platform designed to help professionals and policy makers in the prevention field) suggests that evidence-based prevention programs in school settings that are implemented by properly trained professionals with guarantees of compliance, adherence and fidelity to intervention components, and proper levels of participant exposure are virtually non-existent. The available accounts are rather descriptive and process evaluations (e.g., participant satisfaction) are much more common than studies that evaluate program efficacy. Several factors have been offered in explanation for this, mainly at the organizational level. They include lack of financial resources, leadership, and the institutional culture (low priority given to extra-curricular content) towards evaluation (Moore et al., 2022; Petras et al., 2021).
Prior reviews of school-based prevention programs have shown that the most effective approaches target social and decision-making skills, social norms, and problem-solving skills (Botvin, 2000; Espada et al., 2015; Newton et al., 2022; Onrust et al., 2016). These studies have concluded that competence enhancement approaches are effective in reducing tobacco, alcohol, and marijuana use (and also in other lifestyle and mental health areas). In this regard, efficacy assessments are necessary so that less effective, iatrogenic practices currently in place in school settings can be replaced (Newton et al., 2022).
In a meta-analysis of Spanish school-based prevention programs, Espada et al. (2015) called for more rigorous evaluations and careful supervision and control of the fidelity and integrity of program implementation. Similarly, the National Science Prevention Coalition (2019) recognizes the usefulness and cost-benefit of implementing preventive health practices. These considerations map onto the objectives, procedures, and methodological approach considered in this evaluation protocol.
Internationally, the utility and cost-effectiveness of universal prevention in school settings has been acknowledged in multiple studies (Lee et al., 2017; Levin & Chishold, 2016). Unfortunately, most prevention programs produce small effects on substance use and to further improve effectiveness there is a need to identify robust moderators of behavioral change. In addition, meta-analyses of prevention effectiveness have concluded the need to conduct more rigorous evaluations, including fidelity and integrity of measures to ensure efficacy is properly assessed in prevention practices (Throuvala et al., 2019). In this paper, we report on an e-assessment comprising a wide number of measures that is expected to facilitate nuanced analyses of potential mediators and moderators of change, including specific and non-specific substance-use variables. The inclusion of transdiagnostic variables is important in terms of prevention (Simpson et al., 2022). Research on the determinants of addictive behaviors has led researchers to the proposition that interventions should take a broader approach to risk behavior prevention, targeting not only specific substance-use risk factors (normative beliefs, refusal skills), but also non-specific transdiagnostic targets (emotional regulation) (Kim et al., 2020). Similarly, prevention research has led to the conclusion that evaluation of programs should collect and report multiple health risk behaviors (MacArthur et al., 2018). In addition, these interventions must be evidence-based, accessible, and have protocols in order to facilitate widespread implementation.
This design of evaluation is subject to at least two limitations. First, given that substance use measures are self-reported, some potential bias in substance use estimations is expected. Nonetheless, self-reporting is deemed suitable and has been used in previous evaluations of universal prevention programs in schools (Gaete et al., 2022; Newton et al., 2022). Secondly, the participating schools will not be randomized, instead they will be selected from those that are currently delivering the intervention, limiting the generalizability of results, but ensuring ecological validity.
This study describes the design of the evaluation of the universal school-based prevention program “Juego de Llaves” in Spain. Following quality standards in prevention science, academics (university researchers) and implementers (Asociación Proyecto Hombre), with external support from institutional representatives (UNODC), were all involved in the design of the evaluation. Results from this project will indicate the sustainability of the “Juego de Llaves” program. The authors hope this report will serve as inspiration to improve the culture of evaluation in Spain and facilitate the transfer of knowledge to the community along with reproducibility.
