Introduction

The evaluation of the training of health science specialists in accredited training units and centers is regulated by Royal Decree 183/2008¹. The development and implementation of the royal decree leeds to a Teaching Quality Management Plan, conceived to ensure that training programs meet the requirements set out by the Hospital Pharmacy Specialty Committee².

The Tutors Working Group of the Spanish Society of Hospital Pharmacists (SEFH), made up of both tutors and residents, was created in 2016 with the aim of assisting instructors in discharging their functions more efficiently. The Group’s goals include encouraging the formulation of proposals and development of projects that could improve the quality of training of specialist pharmacists in the future.

Satisfaction surveys of residents provide information regarding residents’ opinion about the quality of the training they receive^3-5. These surveys are useful in that they help detect potential shortcomings in training programs, identify areas of improvement and establish corrective measures. Such surveys are normally organized at a local level (by teaching units or accredited centers), at a regional level, or at a national level by the Ministry of Health, Consumer Affairs and Social Welfare⁶.

In 2002, a survey was supplied among both specialist hospital pharmacists and residents to gain an in-depth understanding of their perception about the training imparted in pharmacy departments and the possibility to successfully implement the contents of the fourth-year residency program. According to the survey, factors influencing the quality of training included the availability of material and human resources, the motivation of tutors and staff pharmacists, and the importance given to the training programs by the department⁷.

The purpose of the present article is to analyze the satisfaction of fourth-year hospital pharmacy residents with the training they receive and identify potential areas of improvement.

Results

Ninety-one of the 135 fourth-year residents concluding their training in 2018 filled out the questionnaire (67.4% of the sample). After reviewing the questionnaires, a decision was made to include all of them in the study.

The demographic characteristics of the surveyed population are presented in Table 1. All respondents were SEFH members; 70% were female and 91% were aged from 26 to 30 years. Of the 91 questionnaires returned, 49 corresponded to fourth-year residents from hospitals with ≥ 7 residents in total, and 42 corresponded to residents from hospitals with < 7 residents.

Table 1.  Demographic characteristics of the surveyed population 

SEFH: Spanish Society of Hospital Pharmacists.

Table 2 shows respondents’ satisfaction, both overall and with the different specific aspects of their residency. Mean overall satisfaction was acceptable-good (3.52 ± 0.92).

Table 2.  Satisfaction of the surveyed population with different aspects of their residency 

Grades were as follows: 5 “very satisfied”, 4 “satisfied”, 3 “acceptably satisfied”, 2 “dissatisfied” y 1 “very dissatisfied”.

SD: standard deviation; SEFH: Spanish Society of Hospital Pharmacists.

A total of 76.9% of residents rated the welcome they received on arrival to the Pharmacy Department on their first year as good-very good. Eighty-nine percent had a good understanding about resident training was organized at their hospital. Satisfaction regarding the usefulness of the hospital’s intranet/educational website was acceptable (3.00 ± 1.02). Access to resources was rated as good (4.04 ± 0.89). As many as 86.8% of respondents had an individualized training plan and 50% believed that their training plan was well or very well attuned to the characteristics of the specialty. A total of 83.5% of residents had been assigned a training program and 79.1% had been included in a rotations schedule; 41.8% had received information on on-call shifts, 27.5% on research activities and 73.6% on training activities.

Seventy-two percent of surveyed residents were satisfied with the usefulness of the hospital’s intranet and/or educational website and 92% were satisfied with the access they were granted to such resources as libraries, journals, books and databases.

When asked about the tutoring received, 63.7% rated the work of their main tutor positively and 72.5% gave positive marks to the tutoring work of other staff members. A total of 42.8% of respondents had at least three annual meetings with their tutor. Nearly 30% (29.7%) were not informed about the criteria used to evaluate their performance and 35.2% were not given information about the grades obtained every year. Residents who did receive this information obtained it from multiple sources: 38.5% from their tutor, 18.7% from the teaching committee, 4.4% from the head of studies, and 3.3% from some other staff member.

Most respondents stated that they recorded the different clinical, educational and research activities in their resident’s logbook or in an annual report.

As far as rotations were concerned, 61.5% of respondents considered that the degree of fulfilment of their rotations schedule was good or very good, and 75.8% stated that the supervision they received from staff members was acceptable. When asked whether their department had a supervision and progressive empowerment protocol, 15.4% answered in the affirmative, 19.8% knew it existed but nobody had explained it to them, 23.1% answered negatively, and 41.8% was not aware that it existed.

Most clinical rotations were completed in internal medicine (73.6%), oncohematology (70.3%), intensive care (57.1%), antibiotic stewardship (56.0%) and pediatrics (47.3%). As many as 90.1% of residents rated the training received in those clinical areas as positive. Tutors in 35.2% of rotations were pharmacists and participated in the evaluation process 34.1% of the time. A total of 70.3% (63/91) of residents did clinical rotations in other hospitals, including 10 rotations in international centers.

As regards on-call shifts, although significant disparity was observed in the number and type of on-call shifts worked, most of them were worked in the afternoon on weekdays and in the morning or afternoon on weekends. Eighty-one percent of respondents considered the level of responsibility assigned to them during their on-call shifts appropriate and 69.2% believed that they had received adequate supervision. Their supervisor during their on-call shifts could either be physically present (25.3%), working remotely (25.3%) or both (partly on site and partly off-site) (49.4%).

When asked about the time devoted to clinical work (including on-call stints) during their work day, 53.8% of respondents said they devoted 30-35 hours a week and 28.6% said they devoted more than 35 hours a week, which means that 96.7% had less than 10 hours a week for educational or research activities. A total of 82.4% of respondents participated in at least three sessions a year, with 69.2% considering that an adequate number. The anticipation with which the sessions were convened was thought appropriate by 91.2% of respondents, while 84.6% said they were satisfied with the quality of the sessions organized.

As for the cross-specialty scheme organized by each hospital, 70.3% of respondents were satisfied with the activities organized; and as regards SEFH-sponsored activities for residents, 74.7% considered them good or very good.

When asked about research, 69.2% of respondents claimed that the information received on research activities was adequate, while 52.7% stated that they received the necessary encouragement and support to participate in them. A total of 35.2% of respondents had produced five or less publications or oral communications for congresses as first authors; 30.8% had completed (or were working on) their PhD dissertation during their residency, and 45.1% were involved in a research project.

Lastly, 89% of respondents were positive about the training received in their hospital, and 75.8% of them claimed they would choose the same hospital again.

Table 3 and Table 4 present the results of a bivariate and multivariate analysis of demographic, training and satisfaction variables, related with overall satisfaction. The analysis revealed a statistically significant association between overall satisfaction and the following variables: supply of information on-call shifts; supply of information on research activities; availability of a tutor during rotations; on/off-site availability of a staff member during on-call shifts; participation in research projects; welcome received on arrival; usefulness of hospital’s intranet/educational website; attunement of the residents’ individualized training plan to the characteristics of the specialty; work of the main tutor; tutoring provided by other staff members; frequency of structured interviews; compliance with rotations schedule; supervision by staff members during rotations; training provided during rotations; empowerment during on-call shifts; service sessions; in-hospital cross-specialty training activities; and SEFH-sponsored training activities. The multivariate analysis performed showed that the work done by the main tutor was independently related with overall satisfaction.

Table 3.  Bivariate and multivariate analysis of the association between demographic and training variables, and resident satisfaction factors 

aSupervision and progressive empowerment protocol. bAs first author.

CDP: cross-discipline plan; CI: confidence interval; Ora: adjusted odds ratio; ORc: crude odds ratio; TP: training program.

Table 4.  Bivariate and multivariate analysis of the association between different variables and overall satisfaction 

CI: confidence interval; Ora: adjusted odds ratio; ORc: crude odds ratio; SEFH. Spanish Society of Hospital Pharmacists.

Discussion

Garnering feedback from specialist trainees is a prerequisite for improving the quality of any health system^3,7. The percentage of responses in our sample (67.4%) is within the range reported in the literature (50-90%)^3,4,8. Previous reports have usually related the percentage of responses received with whether questionnaires were self-refilled or not, and with the respondents’ motivation level. Such methodologies might introduce a bias and lead to erroneous conclusions.

In the present study, the respondents’ overall satisfaction was high, as was their satisfaction with the training they received. Moreover, a high percentage of residents said that they would choose the same hospital if they had to start their residency again.

Individual training plans are prepared by each resident’s tutor based on the specialty’s official curriculum, taking into consideration each hospital’s characteristics and the needs and interests of every resident. This plan makes a huge contribution to the residents’ expertise as it allows tutor and resident to jointly define the competencies to be acquired, the mechanisms to be used in the evaluation process, and the level of supervision to be enforced. The tutor is a key element in the learning process^1,9, a finding that has been borne out by the present paper. Although the tutoring provided by both the main tutor and other staff pharmacists contributed to overall satisfaction, only that provided by the main tutor was found to do so in a statistically significant way.

One of the findings of this study was that 29.7% of residents were not aware of the criteria they were evaluated on. To help tutors with the evaluation process, in May 2018 the SEFH´s Tutors Working Group issued a document titled Herramientas de evaluación para residentes en Farmacia Hospitalaria (Evaluation tools for hospital pharmacy residents)¹⁰, which discusses all the possible evaluation criteria that may be used.

The study also shows that 35.2% of residents were not informed about the grades they obtained every year. For training to be effective, residents must receive feedback from their tutors on whether the goals established at the different tutor-resident interviews have been achieved¹¹. This allows residents to progress faster in their training program and obtain a favorable evaluation. In this respect, SEFH´s Tutors Working Group carried out a Spain-wide survey addressed to resident tutors¹² and detected a need to train tutors on how and when to provide feedback.

Royal Decree 183/2008¹ establishes the need to create a resident supervision system for different specialties, specifying the techniques and practices to be developed and the level of empowerment to be provided at each year of residency. In the survey conducted for the present study, 75.8% of residents considered that the supervision provided by staff members was acceptable; 15.4% had been assigned a supervision protocol and had been progressively empowered to take on more responsibilities. In view of these findings, SEFH´s Tutors Working Group published a guide that provided pharmacy units with a model they could use to establish their own resident supervision protocols¹³.

Addition of a fourth year to the Hospital Pharmacy residency program in 1999, heralded a new era in the training of residents. Indeed, pharmacy departments would thereafter be increasingly involved in clinical pharmacy and pharmacists would increasingly become integrated in multidisciplinary clinical teams. These changes have slowly taken hold and nowadays most pharmacy residents do structured rotations in medical units. According to the survey, only 35.2% of residents stated that the staff member in charge of supervising their rotations was a pharmacist. This means that it was normally a staff member from the relevant clinical area that acted as a supervisor in clinical rotations, which indicates that pharmacists are not fisically in a significant number of clinical units.

Encouragement to do research or produce publications was the area where residents showed the lowest level of satisfaction (2.60 ± 1.28) as compared with overall satisfaction (3.52 ± 0.92). This is a finding confirmed by other authors^5,14,15. The causes of this lack of encouragement include the growing clinical workload healthcare providers are required to shoulder, which limits the time they can devote to other activities. This is borne out by our study, where 96.7% of residents stated that they had less than 10 hours a week available for educational or research activities. This is also connected to the fact that only 30.8% of residents had completed (or were working on) their PhD dissertation. Research is an activity contemplated by all official training programs and all hospital departments wishing to be on the cutting edge of their specialty should ensure that research is integrated within their training programs.

Although the questionnaire has not been validated, it does contain questions included in multiple previously published surveys.

The results of this questionnaire can be used to detect areas for improvement in pharmacy departments, and help tutors identify priorities and share results with other centers.

Overall satisfaction with the training received by fourth-year hospital pharmacy specialist trainees is acceptable. The aspects where respondents expressed lower satisfaction levels had to do with the tutoring provided by their main tutor and other staff members, although only the former reached statistical significance. Areas that should be improved in the future include supervision, the role of staff members in clinical rotations and research. SEFH´s Tutors Working Group will work on the aspects mentioned with a view to improving the quality of training received by hospital pharmacy residents.

Contribution to the scientific literature

The need to understand the way future specialists in hospital pharmacy were being trained led the Tutors Working Group of the Spanish Society of Hospital Pharmacists to prepare a questionnaire to be filled by the fourth-year residents in 2018.

The literature search conducted in preparation for drafting the questionnaire revealed the absence of publications on the subject. Only one study was identified that shared the results of a satisfaction survey administered among hospital pharmacy residents, hospital pharmacists and managers prior to the implementation of a new hospital pharmacy training program that included the addition of a fourth year to the resi dency program. The publication emphasized the importance of making available suitable instructors and adapting the current infrastructure but made no assessment of the new training program or the residents’ satisfaction with its implementation.

This paper seeks to analyze the current situation regarding the training of hospital pharmacy residents, with specific emphasis on their level of satisfaction with the training program they receive and with the instructors imparting it. It also includes an analysis of the available resources and an identification of areas for improvement.

The Tutors Working Group intends to focus their future work on addressing the areas for improvement identified in the study.