Search strategy

The question formulation is an important step to begin a systematic review and it should take into account relevant elements of the research design. For this study, a systematic interpretive review was drafted, focused mainly in qualitative work, to get a scientific-humanist perspectives, and results interpretation (^{Fernández-Ríos & Buela-Casal, 2009}), and three research questions were formulated: Which were the findings related to TPSR implementation in after-school time settings? What research methodologies have been used to examine TPSR in after school time settings? What TPSR-related outcomes were experienced by students in after school time settings?

For the theoretical basis of the present study, articles retrieved from both the TPSR Alliance website and the PsycInfo database were assessed, as the Assessment of Multiple Systematic Reviews (^{Shea et al., 2007}) postulates that at least two different sources should be used.

For the search from the TPSR Alliance website, the reference list updated in January of 2016 was used. This reference list contains Peer-Reviewed Academic and Professional Articles on TPSR and Other Publications Related to or Supporting TPSR An additional search was conducted using PsycInfo, on April 13, 2017. The keywords used in the search systems of the database were: "teaching personal and social responsibility" OR "responsibility model" OR "personal and social responsibility program" AND "after school".

The term "after school" was defined because it is a broader term that includes extracurricular activities (such as activities organized and structured by school teachers) and other activities in or for the community, such as clubsand leadership programs, which are very common in school implementations of the TPSR model (^{Martinek, 2016}).

The search was narrowed to articles between 2001 and 2016, with full text and peer review. Articles retrieved from the PsycInfo database during search procedures were exported to Endnote (electronic reference software) (^{Endnote x7, 2014}).

The selection process of studies for eligibility and data extraction, as suggested by the Cochrane Collaboration (Figure 1) was undertaken by two independent researchers in order to avoid result bias during the selection process (^{Higgins & Green, 2011}). Any disagreements between researchers were discussed and a consensus was reached. Thus, the following selection criteria were applied a priori: a) using the TPSR model in the intervention/ impact on children and youngsters; b) peer-reviewed articles; c) the articles must include participants of TPSR model interventions in after-school contexts; d) articles published between 2001 and January 2016. The exclusion criteria were: a) intervention conducted in a physical education context; b) no access to the full text; c) grey literature (thesis, book, book chapters), as this type of literature is not submitted for peer-review; d) articles that only described strategies of TPSR in after-school contexts.

Figure 1. Flow diagram of literature search 

The degree of agreement was calculated by Cohen's Kappa .83, p< .001 (^{Siegel & Castellan, 1988}), exhibiting an almost perfect agreement (^{Landis & Koch, 1977}).

Score of studies

The criteria for the measured score of the 27 papers included studies adapted from the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement (^{Vandenbroucke et al., 2007}) and the Consolidated Standards of Reporting Trials (CONSORT) statement (^{Moher, Schulz & Altman, 2001}).

Question 1: Did the study provide a detailed description of the program implementation? ‘0' not included; ‘1'description included but it is brief and imprecise (e.g., format plan, duration, included information about participants but did not report details about activities); ‘2' detailed description of the activities was included.

Question 2: Did the study report the duration of the intervention? ‘0' it was not reported; ‘1' nine or less sessions; ‘2' between 10 and 12 or more sessions.

Question 3: Did the paper report validity and reliability. ‘0' not reported; ‘1' in quantitative studies was shown the validity and/ or reliability of the instruments. In qualitative studies was shown the validity through at least one type of triangulation (triangulation of methods, sources, triangulation of the researchers, and triangulation of the theory/perspective). ‘2' in quantitative studies described the steps to the validity and reliability supported in the literature. In qualitative studies was shown the validity through some sort of triangulation with the detailed description of the process of triangulation.

Question 4: Did the study report methodological design and analysis? ‘0' not reported; ‘1' in quantitative studies was shown the methods (questionnaire, direct observation), design (pre- and post-test) and the type of statistics used. In qualitative studies was shown the methods (interviews, focus group, case study, observation, documents) used in the study; ‘2' in quantitative studies was made a detailed description of the methods (description of the instrument), described the purpose of the application of statistics used in the study. In qualitative studies have been made description of the units of analysis (participants, groups), and a description of the category approach (deductive and inductive).

Question 5: Did the paper report findings? ‘0' no findings reported; ‘1' reported findings from authors but did not report findings from all data sources; ‘2' reported findings from authors and all other data sources (e.g., from interviews and questionnaires).

All questions were rated from ‘0' to ‘2', depending on the criteria, shown below in Table 1. The maximum score is 10 points.

Table 1. List of included studies with scores 

Which were the findings related to TPSR implementation in after school time settings?

Duration of the program

At least nine activity sessions are required for intervention in order to improve and facilitate the transfer (^{Catalano, Berglung, Ryan, Lonczak, & Hawkins, 2004}; ^{Petitpas, Cornelius, Raalte, & Jones, 2005}). In the present study it was found that most of the interventions 15/27 fulfilled this assumption (^{Beale, 2012}; ^{Bean & Forneris, 2015}; ^{Buckle & Walsh, 2013}; ^{Forneris et al., 2013}; ^{Hayden et al., 2012}; ^{Hellison & Wright, 2003}; ^{Martinek et al., 2001}; ^{Martinek & Schilling, 2003}; ^{Schilling, 2001}; ^{Walsh, 2007}, ²⁰⁰⁸; ^{Walsh et al., 2010}; ^{Wright et al., 2012}; ^{Wright et al., 2012}; ^{Wright & Gaebler-Spira, 2004}). Still, there were 8/27 who performed interventions under nine sessions (^{Buchanan, 2001}; ^{Hammond-Diedrich & Walsh, 2006}; ^{Lee & Martinek, 2009}, 2012; ^{Newton et al., 2006}; ^{Whitley, 2011}; Whitley & Gould, 2011; ^{Wright, 2012}). However, only ^{Whitley, 2011} reported the five sessions of intervention were not sufficient to provide constructive criticism or to make an impact on the welfare (^{Buchanan, 2001}).

Some studies (3/27) did not report the duration of the intervention (^{Coulson et al., 2012}; ^{Martinek et al., 2006}; ^{Schilling, 2007}).

Intervention location

Most after-school interventions based on the TPSR model (25/27) were conducted in the United States of America (^{Bean et al., 2015}; ^{Beale, 2012}; ^{Buchanan, 2001}; ^{Buckle & Walsh, 2013}; ^{Coulson et al., 2012}; ^{Forneris et al. 2013}; ^{Hammond-Diedrich & Walsh, 2006}; ^{Hayden et al., 2012}; ^{Hellison & Wright, 2003}; ^{Lee & Martinek, 2012}; ^{Lee & Martinek, 2009}; ^{Martinek & Schilling, 2003}; ^{Martinek et al., 2006}; ^{Martinek et al., 2001}; ^{Newton et al., 2006}; ^{Schilling, 2001}; ^{Schilling, 2007}; ^{Walsh, 2007}, ²⁰⁰⁸; ^{Walsh et al., 2010}; ^{Whitley & Gould, 2011}; ^{Wright, 2012}; ^{Wright et al., 2012}; ^{Wright et al., 2012}; ^{Wright et al., 2004}).

However, it was possible to find few interventions with the TPSR model in other developed countries such as Canada, Nepal and South Africa (^{Bean & Forneris, 2015}; ^{Forneris et al., 2013}; ^{Whitley, 2011}).

Participants profiles (from sample)

In this study, young people who teach physical activities to other young people and/ or children were called staff for being in the role of teachers but not being one, and to distinguish the name youngster, the young participants and children who received staff instruction.

Most youngsters (9/27) were considered vulnerable youngsters, due to drug abuse, use of violence, among other behaviors that may lead to social exclusion (^{Bean & Forneris, 2015}; ^{Buckle & Walsh, 2013}; ^{Hellison & Wright, 2003}; ^{Martinek & Schilling, 2003}; ^{Walsh et al., 2010}; ^{Whitley, 2011}; ^{Wright et al., 2012}; ^{Wright, 2012}; ^{Wright et al., 2012}). The staff (10/27) were volunteers attending high school who willingly took part in intervention programs for the community (^{Hayden et al., 2012}; ^{Lee & Martinek, 2009}, ²⁰¹²; ^{Martinek & Schilling, 2003}; ^{Martinek et al., 2006}; ^{Martinek et al., 2001}; ^{Schilling, 2001}, ²⁰⁰⁷; ^{Walsh, 2008}; ^{Hammond-Diedrich & Walsh, 2006}).

Regarding gender, there is a predominance (13/27) of interventions including both female and male (^{Beale, 2012}; ^{Forneris et al., 2013}; ^{Hammond-Diedrich & Walsh, 2006}; ^{Hayden et al., 2012}; ^{Lee & Martinek, 2009}; ^{Lee & Martinek, 2012}; ^{Martinek et al., 2006}; ^{Newton et al., 2006}; ^{Schilling, 2001}; ^{Walsh, 2007}; ^{Walsh, 2008}; ^{Walsh et al., 2010}; ^{Whitley, 2011}). However, there have been studies (3/27) targeted towards either one of genders, female (^{Bean & Forneris, 2015}; ^{Schilling, 2007}; ^{Wright et al., 2012}) or male 5/27 (^{Buckle & Walsh, 2013}; ^{Coulson et al., 2012}; ^{Wright, 2012}; ^{Wright et al., 2012}; ^{Wright et al., 2004}).

Some studies (5/27) did not report the genders of their participants (^{Buchanan, 2001}; ^{Forneris et al., 2013}; ^{Hellison & Wright, 2003}; ^{Martinek & Schilling, 2003}; ^{Martinek et al., 2001}).

As for age, most interventions (11/27) focused on ages between 10 and 14 years (^{Bean & Forneris, 2015}; ^{Forneris et al., 2013}; ^{Hammond-Diedrich & Walsh, 2006}; ^{Hellison & Wright, 2003}; ^{Lee & Martinek, 2012}; ^{Newton et al., 2006}; ^{Schilling, 2001}; ^{Walsh, 2008}; ^{Walsh et al., 2010}; ^{Wright, 2012}; ^{Wright et al., 2012};). In some studies, the age of the participants ranged between 14 and 17 years (High school) (^{Beale, 2012}; ^{Martinek & Schilling, 2003}; ^{Martinek et al., 2006}), or younger ages, such as participants in elementary school (4/27) (^{Lee & Martinek, 2009}; Martinek et al., 2001; ^{Walsh, 2007}; ^{Wright et al., 2012}). Only two studies reported adult participants (^{Coulson et al., 2012}; ^{Schilling, 2007}). Other studies reported several ages at the same time, from 12 to 17 years (^{Buckle & Walsh, 2013}); from 4 to 11 years (^{Wright et al., 2004}), from 10 to 19 years (^{Forneris et al., 2013}) or from 13 to 18 years (^{Forneris et al., 2013}).

Some authors did not specify ages, referring to their participants as "Youngsters" in their studies (^{Buchanan, 2001}; ^{Hayden et al., 2012}; ^{Whitley, 2011}).

When studies reported an intervention within the after school programs, the number of children was higher. However, it is important to note that the number of participants is directly linked to the number of staff who implemented the program. For example, in the study by ^{Hayden et al. (2012}), in which 110 youngsters were distributed into four groups (approximately 29 children in each group), with 12 trainers and 8 university students (staff) responsible for the implementation of the program, who, in turn, were distributed among the four groups, leading to five to eight children for each responsible technician, which literature considers a small group (a maximum of 15 to 20 participants per group) (^{Schilling, 2001} p. 356).

Other studies about after-school programs (8/27) maintained a small group of participants (^{Hammond-Diedrich & Walsh, 2006}; ^{Lee & Martinek, 2009}; ^{Lee & Martinek, 2012}; ^{Martinek & Schilling, 2003}; ^{Martinek et al., 2006}; ^{Martinek et al., 2001}; ^{Schilling, 2001}; ^{Walsh, 2008}). This was also evident in interventions aimed towards vulnerable youngsters (7/27) (^{Bean & Forneris, 2015}; ^{Walsh, 2007}; ^{Walsh et al., 2010}; ^{Whitley, 2011}; ^{Wright et al., 2012}; ^{Wright, 2012}; ^{Wright et al., 2012}) or children with disabilities (^{Wright et al., 2004}), with the exception of the studies (^{Beale, 2012}; ^{Buchanan, 2001}; ^{Hellison & Wright, 2003}), which reported larger sample sizes.

Intervention background

The fact that this research was directed towards after school interventions allowed us to observe a certain heterogeneity among the studies. The interventions as after school programs for community (leadership) where the students from high school prepared activities for children from disadvantaged communities that provided meaningful experiences in personal, social and sports settings for underserved children.

In most studies, the sample size agrees with ^{Hellison (2003}) and ^{Schilling (2001}); in other words, a small sample size (maximum of 20 participants). In case of an excessessive number of participants, they were divided into small groups during the intervention.

Often (9/27), the evaluation of the impact of the TPSR model was mainly focused on the staff (^{Hayden et al., 2012}; ^{Hammond-Diedrich & Walsh, 2006}; ^{Lee & Martinek, 2012}; ^{Martinek & Schilling, 2003}; ^{Martinek, Schilling, & Hellison, 2006}; ^{Martinek et al., 2001}; ^{Schilling, 2001}; ^{Schilling, 2007}; ^{Walsh, 2008}) and seldom on the participants (^{Lee & Martinek, 2009}).

Club activities are another kind of intervention, these sports activities were normally aimed towards vulnerable youngsters (^{Buckle & Walsh, 2013}; ^{Forneris et al., 2013}; ^{Hellison & Wright, 2003}; ^{Whitley, 2011}), and the activities consisted of clubs focusing on basketball (Coaching Club) (^{Walsh, 2007}; ^{Walsh et al., 2010}) and martial arts (Kung Fu Club) (^{Wright, 2012}; ^{Wright et al., 2012}), which took place over several months.

Summer Camps were types of programs characterized by lasting a short period of time, with a significant number of daily hours, and being aimed towards vulnerable youngsters. This type of intervention resorted to the use of fitness and team sports (^{Bean & Forneris, 2015}; ^{Buchanan, 2001}), sets of activities including golf, tennis, swimming, softball and soccer (^{Newton et al., 2006}) or, simply, different types of physical, after-school activities (^{Wright et al., 2012}).

In other contexts, the TPSR model was also found to be adapted to very particular backgrounds, such as a Life Guard course with a certificate at the end (^{Beale, 2012}); associated to youngsters with special needs, such as in the context of recreational therapy (^{Coulson et al., 2012}), and in the context of spastic diplegia cerebral palsy (^{Wright et al., 2004}).

What research methodologies have been used to examine TPSR in after-school time settings?

Type of methodology

The use of a mixed methodology that combines qualitative and quantitative methods is the most recommended in literature for this type of studies and interventions (^{Gorard & Makopoulou, 2012}), as observed in 7/27 of the studies (^{Hayden et al., 2012}; ^{Hellison & Wright, 2003}; ^{Martinek et al., 2006}; ^{Walsh, 2007}; ^{Walsh, 2008}; ^{Wright, 2012}; ^{Wright et al., 2012}). Most of the studies analyzed were qualitative (12/27) (^{Bean et al., 2016}; ^{Beale, 2012}; ^{Buchanan, 2001}; ^{Hammond-Diedrich & Walsh, 2006}; ^{Lee & Martinek, 2009}; Lee & Martinek, 2012; ^{Martinek & Schilling, 2003}; ^{Martinek et al., 2001}; ^{Schilling, 2001}; ^{Schilling, 2007}; ^{Walsh et al., 2010}; ^{Wright et al., 2004}) and only two studies used a quantitative methodology (Bean & Forneris, 2015; ^{Newton et al., 2006}).

In some studies, there was no mention to the methodology used (6/27) (^{Buckle & Walsh, 2013}; ^{Coulson et al., 2012}; ^{Forneris et al., 2013}; ^{Whitley, 2011}; Whitley & Gould, 2011; ^{Wright et al., 2012}).

The lack of scientific rigor in study methodology can be verified (Q 3, Table 1) by the large number of studies that failed to report the validity and reliability (15/27) or just mentioned without great detail (6/27), as well as by the absence or lack of methods (6/27) and a detailed description of the methodology used in the studies (9/27).

What TPSR-related outcomes were experienced by students in after school time settings?

Outcomes

Several studies (10/27) mentioned improvements of personal and social skills, self-control, leadership. However, they didn't report how was data operationalized (^{Beale, 2012}; ^{Buckle & Walsh, 2013}; ^{Coulson et al., 2012}; ^{Martinek et al., 2006}; ^{Walsh, 2007}; ^{Wright et al., 2012}; ^{Wright et al., 2012}; ^{Whitley, 2011}; ^{Forneris et al., 2013}; ^{Whitley & Gould, 2011}).

The results were grouped according to the characteristics of the analyzed studies, based on concepts intrinsic to the TPSR model, such as personal outcomes, social outcomes, leadership, relationship between youngsters and staff, values, transference and impact on the young people's lives.

Personal outcomes - An improvement in young people's behavior, such as paying more attention in class and in their lives, was reported by ^{Walsh (2007}). In other studies, the personal outcomes were evidenced when ten of the thirteen participants created their own goals within and outside the program (^{Walsh et al., 2010}), as well as, when the young people complied with the pre-established goals (^{Martinek et al., 2001}; ^{Schilling, 2007}), thus helping them to cope with dysfunctional problems at home (^{Martinek & Schilling, 2003}), which led to academic improvement for 75% of the participants (^{Wright et al., 2012}).

In the particular case of Tasha (^{Schilling, 2007}), over time, she became more independent and improved her parental functions (teenage pregnancy). Although she may have not fully achieved the personal and social skills, there was an improvement described in her own self-reporting and in the view of the researchers, in terms of her behavior and attitudes towards herself and others.

On the other hand, there was also a decrease of personal responsibility, although it was not statistically significant, at the end of the intervention (^{Bean & Forneris, 2015}). Another study (^{Walsh, 2007}), which compared youngsters development outcomes between young people who participated in the TPSR-based program and students from the same school who had not participated in the program, showed that students who had not participated in the program reported (statistically significant, p<0.05) school being an even greater challenge, when they were approached regarding high expectations.

Social outcomes - There were studies that revealed an increase in the perception of social responsibility at the end of the intervention (^{Bean & Forneris, 2015}), which was also proven in young people with disabilities (^{Wright et al., 2004}). The participants exhibited self-control (^{Buchanan, 2001}; ^{Hammond-Diedrich & Walsh, 2006}; ^{Martinek et al., 2001}) by improving their behavior at school (62% of participants) and by becoming more mature (^{Hammond-Diedrich & Walsh, 2006}).

Another notable example of the acquisition of social skills was reported in the study by ^{Whitley (2011}), in which the young people at the end of the intervention gave more hi-fives and less negative feedback to their teammates. Care for others and willingness towards team work was also well evidenced in the studies by ^{Martinek et al. (2001}) and ^{Walsh et al. (2010}).

On the other hand, in interventions with a shorter duration, it became apparent that, although young people exhibited some changes in their behavior, they still manifested behaviors such as not respecting their teammates by laughing at others (^{Buchanan, 2001}).

Leadership - Leadership ability is a very present concept in the TPSR Some studies reported a development of this ability (^{Hayden et al., 2012}) as a great opportunity for young people to outperform (^{Martinek & Schilling, 2003}). Often, the concept of leadership was intrinsic to the nature of the intervention program, such as the case of the staff who had to autonomously prepare and teach the sports activities to the children (8/27) (^{Hammond-Diedrich & Walsh, 2006}; ^{Hayden et al., 2012}; ^{Lee & Martinek, 2009}; ^{Lee & Martinek, 2012}; ^{Martinek & Schilling, 2003}; ^{Martinek et al., 2006}; ^{Schilling, 2001}; ^{Walsh, 2008}).

Relationship between staff and youngsters - The relationships established between the staff from university and the staff from high school (4/27) (^{Buckle & Walsh, 2013}; ^{Hellison & Wright, 2003}; ^{Schilling, 2001}; ^{Walsh et al., 2010}), between peers (^{Wright, 2012}), or between staff and youngsters (^{Schilling, 2001}) were highlighted as a very positive aspect of the program, with a mutual benefit (^{Hammond-Diedrich & Walsh, 2006}; ^{Martinek et al., 2006}), regardless of age (^{Hammond-Diedrich & Walsh, 2006}; ^{Wright et al., 2004}). This relationship was strengthened by the care and concern towards youngters' lives, which motivated them not to give up (^{Buckle & Walsh, 2013}) on their goals and made them aware of the importance of taking care of their well-being and the well-being of others (^{Hellison & Wright, 2003}; ^{Martinek & Schilling, 2003}). The strong relationships made a difference in the most underserved children in the community (^{Martinek & Schilling, 2003}).

Values - In the interventions, youngsters stated that the TPSR model was a facilitator of role-modeling behavior, as it was based on values such as respect for the rights and feelings of others (^{Schilling, 2001}). Participants also showed to be more aware of the use of these skills (values) outside the program (^{Wright et al., 2012}). Paul Wright, in his study (^{Wright, 2012}), reported that 70% of participants evidenced improvement regarding certain values such as honesty, care, respect and responsibility.

The participation of youngsters in value-based and task-based programs rather than ego-based programs has generally led to the development of responsibility in staff (^{Hammond-Diedrich & Walsh, 2006}; ^{Newton et al., 2006}).

Transference - It is relevant to point out that the transfer of the values learned in the sessions is grasped and applied outside the context of the session (^{Hellison, 2011}).

Some studies (8/27) focused on the theme download, identifying benefits and results as the transference of the values learned in the sessions to after school-settings. It was found that 62% of participants demonstrated a medium and high level of transference (^{Martinek et al., 2001}). In other analyzed studies, participants reported that the program would help them both within and outside the intervention (^{Beale, 2012}; ^{Forneris et al., 2013}; ^{Hammond-Diedrich & Walsh, 2006}; ^{Lee & Martinek, 2012}; ^{Walsh et al., 2010}), as these youngsters became better individuals, and not only better basketball players (^{Hellison & Wright, 2003}), since that transference would aid them in decision making, creating aspirations and making choices for the future (^{Walsh, 2008}).

On the other hand, two studies reported that the transference was the least visible level (subject) during the interventions (^{Hayden et al., 2012}), and that culture, for example, due to its characteristics, also functioned as a barrier to the transference of the program values to the school setting, as is evidenced in the study by ^{Lee and Martinek (2009}), where the empowerment values in Project Effort (e.g., respect, responsibility) were perceived as a discipline approach in the school setting.

Although, there is no knowledge in the literature of instruments that measure the perception of transfer, in some cases (6/27) this was considered for interviews (^{Hammond-Diedrich & Walsh, 2006}; ^{Hayden et al., 2012}; ^{Lee & Martinek, 2009}, ²⁰¹²; ^{Walsh, 2008}; ^{Walsh et al., 2010}).

Impact on staff - A positive impact of the program on the lives of the participants, as well as their respective communities, was reported by ^{Forneris et al. (2013}). The staff described the program as safe (^{Whitley, 2011}), fun (^{Hammond-Diedrich & Walsh, 2006}; ^{Forneris et al., 2013}; ^{Schilling, 2001}; ^{Wright et al., 2012}) - even more than traditional therapy (^{Wright et al., 2004}) -, and that the content (martial arts) was in line with TPSR-based values TPSR (^{Wright et al., 2012}; ^{Wright et al., 2004}).

Through the TPSR program, staff improved their teaching skills with children and became more familiar with college (intervention location) (^{Hammond-Diedrich & Walsh, 2006}). Individual experiences during the interventions were reported by staff as very significant when they led a group of children, and when they worked in peer-coaching (^{Hammond-Diedrich & Walsh, 2006}; ^{Wright et al., 2012}), because it allowed them to learn with their peers (^{Forneris et al., 2013}).

Other evidence was reported by staff when they conveyed the desire to spend more time in the program (^{Hammond-Diedrich & Walsh, 2006}; ^{Walsh, 2007}). With assiduous participation in the program, staff reported that it provided them with opportunities to follow through with their commitments (^{Schilling, 2001}; ^{Whitley, 2011}).

In a more longitudinal view, the instructor with experience in leadership programs continued with TPSR-based teaching in her professional life, she was guided by values and was enthusiastic about passing them on to the children after observing positive changes in their behavior (^{Coulson et al., 2012}).