Introduction

Over the last few decades, a considerable amount of research has highlighted the poorer outcomes that care leavers display as young adults in comparison with their non-care-experienced peers. Risks such as low educational attainment, unemployment, housing instability, or a higher vulnerability to suffering mental well-being issues are usually hampering their journeys to independent living (^{Gypen et al., 2017}; ^{Kääriälä & Hiilamo, 2017}; ^{Martín, González, et al., 2020}). However, while states are legally required to provide alternative care and address the needs of children and adolescents as long as they are less than 18 years old, this protection is usually not applicable during their transition to adulthood (^{Stein & Ward, 2021}). In addition, this process is expected to start earlier and take less time to be completed for them, despite the growing prolongation of the normative transition to adulthood in favour of a period of experimentation and choice of different life opportunities named “emerging adulthood” (^{Arnett, 2019}).

The study of pathways of those care leavers that succeed in their transition to adulthood has consistently found experiences that enhance resilience and life-course agency to be crucial in their trajectories (^{Bengtsson et al., 2020}; ^{Lou et al., 2018}; ^{Van Breda & Dickens, 2017}). In this respect, preparation before leaving care is one of the most challenging and vital points to consider to promote successful journeys from care to independent living (^{Harder et al., 2020}). Adequate preparation has to be planned well before leaving care, providing early opportunities for young people to gradually develop independent living skills and promote autonomy in care (^{Armstrong-Heimsoth et al., 2020}; ^{Mendes et al., 2011}). As a result, the last few years have seen an increased interest in developing transitional support services available for young people in care to improve their life skills (^{Woodgate et al., 2017}).

While there is evidence that participating in this kind of independent living programs has some positive effects on post-transition outcomes for care leavers (^{Heerde et al., 2018}), few attempts have been made to investigate the role of the direct outcomes of these interventions for young people in or leaving care, including their perceived ability for independent living or life skills. In fact, a recent systematic review on young people’s preparedness for adult life after foster care found only seven studies that focused on evaluating the abilities of young people in care during the last decade (^{Häggman-Laitila et al., 2019}).

Most of these studies tended to report high and optimistic views of young people about their abilities to live independently (^{Casey et al., 2010}; ^{Dinisman & Zeira, 2011}; ^{Dutta, 2017}; ^{Trout et al., 2014}), which contrast with the consistent findings from international research in which care leavers usually report feeling ill-prepared while they were making the transition to independent living (^{Atkinson & Hyde, 2019}; ^{Bond, 2020}; ^{Courtney et al., 2011}). In this respect, some authors have argued that optimistic evaluations might be due to an overestimation of young people’s own skills when they are based on a prospective assessment (^{Benbenishty & Schiff, 2009}; ^{Casey et al., 2010}), representing their desires and expectations of positive future outcomes, or sometimes highly explained by individual variables such as self-esteem or working experience (^{Dinisman & Zeira, 2011}).

However, it is still difficult to draw firm conclusions from this research for several reasons. First, most studies used only self-reported measures of perceived ability for independent living (^{Häggman-Laitila et al., 2019}). In fact, the only one that included a multi-informant perspective found a gloomier perspective of young people’s abilities when this was reported by their parents (^{Trout et al., 2014}), which is similar to what ^{Refaeli et al. (2013)} found when they examined residential care staff perspectives on this matter. Second, studies rarely use standardised measures to report independent living skills (^{Harder et al., 2011}). Although the use of standardised multi-informant measures in the field of child welfare is becoming more common for assessing areas such as mental health and well-being (^{Martín, González-García, et al., 2020}; ^{Vallejo-Sloker et al., 2020}), international comparison of results is still difficult in the field of assessing readiness for independent living and life skills.

This lack of research is even more pronounced in the Spanish context. Although there were some instruments available to measure independent life skills that had been translated from English to Spanish and included self-reported and caregiver versions, like the Casey Life Skills Assessment (CLSA; ^{Casey Family Programs, 2017}) and the life skills questionnaire included in the Umbrella Program (^{Del Valle & García-Quintanal, 2006}), there was no evidence that supported its validity to be used with Spanish populations nor studies on the level of agreement and relationships between informants’ views.

This gap has been partly addressed with the development and validation of a new set of measures, the PLANEA Independent Life Skills Assessment tools (^{García-Alba et al., 2021}), linked to the framework that the independent life skills training tool Planea Program proposed (^{Del Valle & García-Alba, 2021}). PLANEA instruments have already shown robust psychometric properties and promising opportunities for being used in research with general and care-experienced young populations, given their short length and easy application (^{García-Alba et al., 2021}) compared to the ones previously mentioned. However, the views of the staff regarding the independent living skills of young people in care remained unexplored, as these instruments included only self-reported versions.

Therefore, the present study pursued a double objective. First, to adapt and validate a staff version of the PLANEA Independent Life Skills Assessment tools. Second, to explore the level of agreement and convergence between self-reported and staff assessments of independent life skills and autonomy on a sample of young people living in residential child care. We hypothesize that the adapted staff version of these instruments will replicate the factorial model of the young people versions and present adequate psychometric properties. Second, if the instruments are properly validated, adequate levels of agreement and convergence between youth and staff’s scores will be found.

Method

Participants

In order to validate the staff version of PLANEA Independent Life Skills Assessment tools, a sample of 422 young people living in residential care homes (78%) or engaged in aftercare support services (22%) for care leavers was used. Of those still in care, 25% were living in children’s homes specifically for adolescents preparing to leave care, while the rest were living in basic network homes (46%) or homes for young people with specific needs or profiles (29%). Young people’s independent living skills and personal autonomy levels were evaluated by one of their residential or care workers, who had been consistently working with them as caregivers or educators at the moment of data collection. In most cases, this person was their key residential worker (55%) or other care workers in the home or service (36.4%), being less frequent to be evaluated by the home manager (4.3%) or others (4.3%). A total of 219 residential or care workers participated in the study, being possible that each of them evaluated more than one young person.

In order to study the level of agreement between young people’s and staff’s responses, 392 responses of young people to the self-reported version of the PLANEA Independent Life Skills Assessment tools were possible to retrieve out of the 422 youth evaluated. This sample included 159 females (40.6%) and 233 (59.4%) males from 15 autonomous communities in Spain. Of them, 25.1% were identified as unaccompanied migrant young people. Youth’s ages ranged from 14 to 25 years (M = 17.1, SD = 2.09), who had been in residential child care for 3 years on average (M_months = 39.25, SD = 38.98)

Table 1.  Items of the PLANEA Independent Life Skills Assessment Tools – Staff Version  

Note. ¹Items included in PLANEA-9-S.

Results

Evidence Based on Internal Structure

The CFA of PLANEA-S showed an adequate fit (CFI = .930; NNFI = .922; RMSEA = .064 CI 90% [.061, .068]) in a three first-order factors and one second-order factor model. The factor loading of each item is shown in Table 3, ranging between .390 and .810 in Self-Care and Well-Being, between .508 and .890 in Daily Arrangements and Organisational Skills, and ranging between .480 and .898 in Employment and Accommodation. The factor loadings of the first-order factors to the second-order factor were .825, .969, and .919.

Table 2.  Fit Indices for the Measurement Invariance of PLANEA Instruments Based on the Type of Informant (Young People and Staff)  

Note. χ² = Satorra-Bentler chi-square; df = degrees of freedom; CFI = comparative fit index; ΔCFI = CFI change.

Table 3.  Descriptive Statistics, Factor Loadings, and Reliability of PLANEA-S 

Note. ¹Items in PLANEA-9-S.

Regarding PLANEA-9-S, which is composed by 3 items of each dimension in PLANEA-S (all of them with very high factor loadings; see Table 3), showed a good fit when considering it as unidimensional (CFI = .982; NNFI = 976; RMSEA = .072 CI 90 % [.055, .089]). Furthermore, the factor loadings ranged between .531 and .912.

Finally, PLANEA-T-S showed an excellent fit (CFI = .999; NNFI = .999; RMSEA = .019 CI 90 % [.001, .038]) in a two-factor correlated model. Pearson’s correlation between both dimensions was high (r = .732). The factor loadings, available at Table 4, ranged between .771 and .852 (Managing Daily Life Tasks), and between .523 and .844 (Doing Household Chores).

Table 4.  Descriptive Statistics, Factor Loadings, and Reliability of PLANEA-9-S and PLANEA-T-S 

Once the factor structure of the staff versions was confirmed, we continued with examining measurement invariance between young people and staff versions. The results of this analysis are available in Table 2. Invariance was confirmed at the three levels examined (configural, metric, and scalar) in PLANEA, PLANEA-9, and PLANEA-T.

Discriminative Capacity of the Instruments

To study the discriminative capacity of the staff version of the instruments (PLANEA-S, PLANEA-9-S, and PLANEA-T-S), we performed independent samples t-tests between groups of young people expected to show different levels of autonomy according to their type of placement and case plan. Regarding their type of placement, we compared those young people (36% of the total sample) that were living in group homes from the basic network to those who were living in group homes for adolescents preparing to leave care (19.2%) or in supported accommodation for care leavers (22%). The latter group (46%), named “autonomy”, should feel more skilled and be more autonomous, considering that they had been engaged in interventions and programs for this purpose. The rest of the young people, who were living in specialised residential care homes (e.g., for migrant young people, for young people with severe emotional or behavioural challenges), were not considered in this comparison due to their specific characteristics. Regarding young people’s case plan, we compared those who included the development of autonomy and life skills for emancipation as one of the main objectives of the educational intervention from the program (84.1%) to those who did not (15.9%), as informed by their educators.

Differences were found in all dimensions of the studied instruments (Table 5). Independent living skills and personal autonomy of young people engaged in specific programs or homes for developing autonomy and independent living skills (whether they were still in care or engaged in aftercare support services) were evaluated significantly higher by their care workers, compared to their peers living in children’s homes from the basic network. The same trend was found for those whose intervention case plan included autonomy as a priority regardless of their placement characteristics. However, these results must be interpreted with caution in the case of PLANEA-T, considering the small sample size available in some of the groups (Table 5).

Table 5.  Differences in PLANEA-S, PLANEA-9-S, and PLANEA-T-S Based on Young People’s Level of Autonomy  

Note. M = mean; t = Student’s t statistic; d = effect size.

Level of Agreement Between Informants

The level of agreement between informants was studied for PLANEA and PLANEA-T instruments. The scores were significantly higher in the young people version of PLANEA across all dimensions and main scale than staff’s scores, with medium effect sizes (Table 6). In contrast, the magnitude of this difference is reduced in PLANEA-T, being significant only for dimension 2 (Doing Household Chores) and showing minimal effect size. The correlations between versions were large, significant, and positive for all the studied scales and subscales, being less than .50 in only one of the subscales in PLANEA (Self-Care and Well-Being), and slightly higher for the total scores of PLANEA and PLANEA-T than for their subscales, up to .817 (Table 7).

Table 6.  Differences in Mean Scores Between Versions and Role of Staff Informants in PLANEA, PLANEA-9, and PLANEA-T  

Note. M = mean; t = Student’s t statistic; d = effect size.

Table 7.  Correlations Between Young People and All Staff Scores in PLANEA and PLANEA-T  

Note. Values in bold highlight the correlations between the two informants’ scores in the same scale or subscale.

In addition, an effect related to the role of the respondent to the staff version was found across all scales and subscales studied. Key residential care workers (those with direct responsibility in the supervision of the educational intervention of the young people evaluated) scored higher than the rest of the staff members (other residential care workers or program managers), showing medium effect sizes (Table 6). The correlations between young people’s scores and key residential care workers’ scores (Table 8) were slightly higher than those with non-key residential staff (Table 9), except for the Self-Care and Well-Being subscale. However, none of these differences were significant (p < .05) except for one subscale of PLANEA: Daily Arrangements and Organizational Skills (z = 2.46, p = .014).

Table 8.  Correlations Between Young People and Key Educator Staff Scores in PLANEA and PLANEA-T  

Note. Values in bold highlight the correlations between the two informants’ scores in the same scale or subscale.

Table 9.  Correlations Between Young People and Non-key Educator Staff Scores in PLANEA and PLANEA-T  

Note. Values in bold highlight the correlations between the two informants’ scores in the same scale or subscale.

Discussion

Ample international evidence supports the need to provide young people in care with adequate preparation for leaving care (^{Harder et al., 2020}). However, the obligation for child care agencies to offer preparation for independent living for young people in residential care since they are 16 years old is still a relatively recent addition to Spanish child welfare laws. This update is already impacting the development of new interventions for this purpose, such as Planea Program (^{Del Valle & García-Alba, 2021}), and new assessment frameworks to study perceptions of readiness for independent living in young care-experienced people (^{García-Alba et al., 2021}).

The present study aimed to expand this framework by adapting and validating a staff version of the PLANEA Independent Life Skills Assessment tools and narrowing the gap in knowledge on the effectiveness of transitional support interventions for young people in care and their perceptions of independent life skills. This is especially pronounced in the Spanish context where, in contrast with the consolidated body of evidence-based practice, programs and assessment tools available in other related areas such as family intervention (^{De Paul et al., 2015}; ^{Rodrigo, 2016}), intervention with juvenile offenders (^{Cacho et al., 2020}), or quality evaluation in residential child care (^{Pérez-García et al., 2019}), the specific evaluation of transitional support services is still recent and has limited its scope to gather young people’s and workers’ views and satisfaction (^{Goig & Martínez, 2019}; ^{Sevillano-Monje et al., 2021}).

The results of the CFAs performed in the staff versions of the three instruments studied allowed us to confirm the same internal structure found in the young people versions: three first-order factors and one second-order factor (Independent Life Skills) in the case of PLANEA-S, unidimensionality for its short version PLANEA-9-S, and two first-order factors correlated between them, showing a global score (Personal Autonomy) for PLANEA-T-S (^{García-Alba et al., 2021}). The instruments showed satisfactory psychometric properties with very high reliability coefficients across all subscales and total scores. Furthermore, invariance measurement was also found at the three levels studied (configural, metric, and scalar), demonstrating that the PLANEA instruments reflected the same construct for the examined groups (young people and staff) and that the scores they gave have the same significance for everyone assessed (^{AERA, APA, NCME, 2014}; ^{Thompson, 2016}). This allows us to make valid comparisons and confidently interpret differences in their scores (^{Milfont & Fischer, 2010}).

The comparison of staff’s scores in different levels of expected autonomy in young people brought evidence of the discriminative capacity of the studied instruments. As we expected, the staff informed of clearly higher levels of life skills and personal autonomy for those young people engaged in specific resources for preparation of autonomy and for those whose case plan included this as one of the aims of the educational intervention, compared to those without this condition.

Our study on the level of agreement between young people and staff scores in the instruments yielded some interesting findings. Although the correlations between young people’s and care workers’ responses were significant, positive, and high across all instruments, remarkable differences were found between informants. Regarding independent life skills, young people in our study tended to perceive themselves as more skilled than their residential care workers. These results seem to confirm what has been found in previous studies that have highlighted how young people in care usually have an optimistic perception about their readiness and abilities for independent living, especially in daily living skills (^{Häggman-Laitila et al., 2019}) and tend to hold positive long-term expectations about their future (^{Bengtsson et al., 2018}; ^{Sulimani-Aidan, 2015}). This contrasts with the less optimistic views that staff or parents display in childcare contexts (^{Casey et al., 2010}; ^{Refaeli et al., 2013}; ^{Trout et al., 2014}) and other similar fields, such as juvenile correctional facilities (^{Melkman et al., 2016}). Maybe these views could be considered more realistic, as previously noted by research that preparation for leaving care services is frequently insufficient to meet young people’s needs (^{Bond, 2020}; ^{Burgund et al., 2018}; ^{Pinkerton, 2021}).

Some authors have also hypothesised that differences between informants in this respect might be due to an overestimation of their competence, which is also known as “positive illusory bias” (^{Benbenishty & Schiff, 2009}; ^{Casey et al., 2010}). This effect has been widely found in self-assessments of competence for every-day life activities in children and adolescents with ADHD, and its magnitude seems to be mediated by the presence of externalising behaviour problems, which may increase the probability of the young people receiving negative assessments of their competence from adults (^{Volz-Sidiropoulou et al., 2016}). This issue deserves further research for the case of children and adolescents living in residential child care facilities, considering the high prevalence of externalising behavioural challenges that they tend to show, according to ^{González-García et al. (2017)} and ^{Águila-Otero et al. (2020)}.

On the contrary, this difference was not observed in personal autonomy (PLANEA-T), as young people and staff tended to show similar scores, and young people only scored significantly higher in one of the subscales, showing small effect sizes. This might be related to the fact that this instrument measures the level of autonomy displayed in the performance of every-day life tasks (e.g., cooking meals, making appointments), which are observable and shared in the residential homes and, in contrast with perceived ability, they may be less likely to be explained by individual variables such as self-efficacy (^{Dinisman & Zeira, 2011}). It also supports the conclusions of ^{Casey et al. (2010)} about the importance of using multi-informant assessment approaches combined with more objective measures in this field.

Moreover, an effect on staff’s scores was observed regarding their role towards the young person evaluated. Those care workers with direct responsibility for the educational intervention of the young person seemed to evaluate young people’s life skills and personal autonomy as slightly higher than the rest of the members of staff (e.g., program coordinators, other educators). A possible interpretation of this finding is that key educators may have a more profound knowledge of the young people’s abilities and a more accurate assessment of their potential in the light of their progress. This is supported by the generally higher correlations found between scores of key educators and young people than in the case of other care workers and the young people they evaluated. However, differences were significant only for one subscale of the PLANEA test: Daily Arrangements and Organisational Skills. It is also possible that the staff’s own expectations of success in their intervention with young people might be playing a role, or that a closer relationship with the young person is promoting a less objective vision of their strengths and difficulties, as ^{Casey et al. (2010)} have pointed out. The influence of staff characteristics or role towards the young person evaluated will also need further research, as it had not been addressed comparatively in literature.

The present study is not free of limitations. First, the recruitment procedure used may have overrepresented those young people who were more skilled or motivated to participate. Although confidentiality and anonymity of their responses were guaranteed, young people being presented with the possibility of participating in this study by residential care staff might have a social desirability effect on their responses (^{Ferrando & Navarro-González, 2021}). Second, the effect of potentially relevant individual variables such as self-efficacy or participants’ profile characteristics (e.g., gender, age) on perceived independent life skills and level of autonomy has not been considered in this study. Third, the effect of the specific characteristics of the interventions received by young people preparing for leaving residential child care has not been controlled in this study. These may vary from one region to another and impact on staff perspectives regarding young people’s skills, as there is not a unique detailed legal framework applicable to the whole country (^{Arnau-Sabatés et al., 2021}). Finally, no demographic information was gathered about the staff respondents that allowed to study potential biases in staff assessments regarding their gender, age or seniority in their job.

Future research should further address several issues. First, longitudinal studies should explore the evolution of perceived ability for independent living over time for young people in care and its potential effects on their success and future outcomes. In this respect, ^{Bengtsson et al. (2018)} have highlighted how expectations of care leavers about their future outcomes vary over time. Second, it is especially relevant to study the impact of participating in interventions aimed at the development of autonomy and skills for independent living in young people’s perceived abilities, as previous studies have pointed out that receiving training for independent living might not directly contribute to perceived readiness (^{Dixon et al., 2006}). For this reason, the mediating effect of young people’s personal and individual characteristics, such as self-efficacy or self-esteem, should also be considered, devoting special attention to the impact of experiences that enhance the sense of agency and increase the probability of success for care leavers, such as early working experiences (^{Arnau-Sabatés & Gilligan, 2015}). Third, it is essential to study further the role of staff perspectives in the decision-making processes that affect the provision of support services for young people leaving care.

In conclusion, the present study has introduced a new reliable and valid staff version of the PLANEA Independent Life Skills Assessment tools, which expands the framework for assessing perceived independent life skills and personal autonomy from a multi-informant perspective in the field of child protection. Furthermore, the availability of multiple multi-informant scales and short versions in this framework makes it possible to design tailored assessments that meet the specific requirements of each research study or applied context, optimising application times and facilitating a rich understanding of the variables measured (^{Ruiz-Hernández et al., 2020}).

These instruments will be disseminated along with the Planea Program in different regions of Spain after being piloted, implemented, and evaluated in the region of Castilla-La Mancha. Work is already underway to translate and adapt the PLANEA Independent Life Skills Assessment tools to English-speaking countries and implement them in the agencies that constitute the Latin American Network of Care Leavers (LATAM Network).

Our study of the level of agreement between self-reported and staff views on independent living skills and autonomy of young people in residential child care supports a high convergence between versions and highlights the importance of using cross-informant approaches in this field. This will hopefully help direct the attention to young people’s perceived strengths and difficulties in this area and inform decision-making in the provision of interventions to promote a gradual, planned, and prepared transition to adulthood from care, which remains one of the main challenges faced by child protection agencies nowadays.