ORIGINAL RESEARCH

 

Challenges to publishing pharmacy resident research projects from the perspectives of residency program directors and residents

El reto de publicar proyectos de investigación de los residentes en farmacia desde la perspectiva de loa directores y residentes de programas de residencia

 

 

Adriane N. Irwin¹, Kari L. Olson², Brigitte R. Joline³, Daniel M. Witt⁴ and Rachana J. Patel⁵

¹PharmD. Clinical Pharmacy Research Fellow. Kaiser Permanente Colorado. Aurora, CO (United States)
²PharmD. Clinical Pharmacy Specialist, Kaiser Permanente Colorado; & Clinical Associate Professor, Skaggs School of Pharmacy and Pharmaceutical Sciences, University of Colorado. Aurora, CO (United States)
³Skaggs School of Pharmacy and Pharmaceutical Sciences, University of Colorado. Aurora, CO (United States)
⁴PharmD. Senior Manager, Clinical Pharmacy Services at Kaiser Permanente Colorado; & Clinical Assistant Professor, Skaggs School of Pharmacy and Pharmaceutical Sciences, University of Colorado. Aurora, CO (United States)
⁵PharmD. Residency Supervisor and Clinical Pharmacy Specialist in Primary Care at Kaiser Permanente Colorado; Clinical Assistant Professor, Skaggs School of Pharmacy and Pharmaceutical Sciences, University of Colorado. Denver, CO (United States)

This study was funded internally by the Kaiser Permanente Colorado pharmacy department.

 

 

---

ABSTRACT

Objective: To identify barriers to completing and publishing pharmacy residency research projects from the perspective of program directors and former residents.
Methods: This was a cross-sectional survey of pharmacy residency program directors and former post-graduate year one and two residents. Directors of pharmacy residency programs whose residents present their projects at the Western States Conference (n=216) were invited to complete an online survey and asked to forward the survey to former residents of their program in 2009, 2010, or 2011. The survey focused on four broad areas: 1) demographic characteristics of the residency programs, directors, and residents; 2) perceived value of the research project; 3) perceived barriers with various stages of research; and 4) self-identified barriers to successful research project completion and publication.
Results: A total of 32 program directors and 98 residents completed the survey. The minority of programs offered formal residency research training. Both groups reported value in the research project as part of residency training. Significantly more directors reported obtaining institutional review board approval and working through the publication process as barriers to the research project (46.7% vs. 22.6% and 73.3% vs. 43.0%, respectively p<0.05) while residents were more likely to report collecting and analyzing the data as barriers (34.4% vs. 13.3% and 39.8% vs. 20.0%, respectively, p<0.05). Both groups self-identified time constraints and limitations in study design or quality of the study as barriers. However, while program directors also indicated lack of resident motivation (65.5%), residents reported lack of mentorship or program structural issues (43.3%).
Conclusion: Overall, while both groups found value in the residency research projects, there were barriers identified by both groups. The results of this study may provide areas of opportunity for improving the quality and publication rates of resident research projects.

Key words: Publishing; Research; Education, Pharmacy, Graduate; Internship, Nonmedical; Pharmacists; United States.

---

RESUMEN

Objetivo: Identificar las barreras para completar y publicar los proyectos de investigación de la residencia en farmacia desde la perspectiva de los directores de programas y de los antiguos residentes.
Métodos: Este fue un estudio transversal de directores de programa de residencia y antiguos residentes post-graduados de año 1 y 2. Se invitó a completar un cuestionario online a los directores de programas de residencia cuyos residentes presentaron proyectos en la Western States Conference (n = 216) y se les pidió que pasasen el cuestionario a los antiguos residentes de sus programas de los años 2009, 2010 o 2011. El cuestionario se centraba en cuatro grandes áreas: 1) características demográficas de los programas de residencia, los directores y los residentes; 2) valor percibido del proyecto de investigación; 3) barreras percibidas en los diferentes pasos de la investigación; y 4) barreras auto-identificadas para la conclusión exitosa y publicación del proyecto de investigación.
Resultados: Un total de 32 directores de programas y 98 residentes completaron el cuestionario. Una minoría de programas ofrecía formación formal en investigación. Ambos grupos encontraron valor al proyecto de investigación, como parte de la formación de la residencia. Significativamente más directores comunicó que obtener la aprobación de la comisión de investigación de la institución y el trabajo de la publicación eran barreras para el proyecto de investigación (46.7% vs. 22.6% y 73.3% vs. 43.0%, respectivamente p<0.05), mientras que los residentes reportaban con más probabilidad como barreras la recogida de datos y el análisis como barreras (34.4% vs. 13.3% y 39.8% vs. 20.0%, respectivamente, p<0.05). Ambos grupos auto-identifico la falta de tiempo y las limitaciones en la calidad del diseño del estudio o su calidad como barreras. Sin embargo, mientras que los directores también indicaban la falta de motivación de los residentes (65,5%), los residentes reportaron la falta de tutela o problemas estructurales del programa (43,3%).
Conclusión: En general, mientras que los dos grupos encontraron valor en los proyectos de investigación en la residencia, había barreras identificadas por los dos grupos. Los resultados de este estudio pueden proporcionar áreas de posible mejora de la calidad y las tasas de publicación de los proyectos de investigación delos residentes.

Palabras clave: Edición; Investigación; Educación de Postgrado en Farmacia; Internado no Médico; Farmacéuticos; Estados Unidos.

---

 

Introduction

The pharmacy profession is moving towards producing highly skilled clinicians with advanced clinical training.^1,2 Pharmacy residency programs play an important role in meeting this objective with the number of programs increasing in response to higher demand for more skilled clinicians. To receive accreditation by the American Society of Health-System Pharmacists (ASHP), pharmacy residency programs must provide opportunity for residents to "conduct a practice-related project using effective project management skills".^3,4 Projects can be original research, medication use evaluations (MUE), quality improvement projects, or problem solving exercises.^3,4 Residents typically present their work at regional residency conferences and are often encouraged, although not typically required, to publish their findings in peer-reviewed journals.

Recent studies demonstrate low publication rates of pharmacy resident projects.^5-7 Only 16% of resident projects presented at the Southeastern Residency Conference (SERC) were published.⁵ Similarly, publication rates of projects presented at the Western States Conference (WSC) have been estimated to be between 4.3% and 6.3%.^6,7

Pharmacy directors acknowledge that executing a successful research project is one of the most challenging aspects of residency training.⁸ However, the specific aspects of the research and publication process that are most challenging are not well defined. Identifying these aspects may prove useful for residency programs desiring to improve their research training and project publication rates. The purpose of this study was to identify barriers to pharmacy residency research project completion and publication from the perspective of the program directors and former residents. We hypothesized that perceived barriers to publishing resident research projects from the perspective of the residency director will differ from those of the resident.

 

Methods

Study Design and Participants

This was a cross-sectional survey of pharmacy residency program directors and former residents. Postgraduate year 1 (PGY1) and postgraduate year 2 (PGY2) residents who would have presented projects at WSC in 2009, 2010, or 2011 were eligible to complete the survey. If a resident had completed both a PGY1 and PGY2, then they were asked to answer the survey based on their PGY2 experience. This study was submitted to the institution review board and determined to be exempt from review.

Survey Development and Content

Two surveys specific to the perspectives of program directors and former residents, but which contained similar items for comparison purposes were developed. The surveys focused on four broad areas: 1) demographic characteristics of the residency programs, program directors, and former residents, 2) level of difficulty associated with various stages of research (rated on a Likert-scale: 1=hardly challenging, 5=very challenging), 3) perceived value of the research project(rated on a Likert-scale: 1=strongly disagree, 5=strongly agree), and 4) self-identified barriers to successful research project completion (open-ended response).

Survey Administration

The ASHP residency program directory was used to identify residency directors of programs who would likely attend the WSC. The directors were sent an email containing a short explanation of the study and a link to the online survey (SurveyMonkey, Palo Alto, CA). Three additional emails were sent at seven to ten day intervals to non-responders to encourage completion of the survey. Program directors were asked to forward the email to their former residents who had completed a residency between 2009 and 2011. This timeframe was chosen as it allowed residents at least 12 months to submit their project for publication while limiting the potential for recall bias.

Statistical Analysis

Percentages were used to summarize categorical data. Likert-scale responses were dichotomized by grouping answers 1 through 3 and 4 and 5. Differences in categorical variables between program directors and former residents were compared using Pearson´s chi-squared test of association or Fisher´s exact test for small numbers. Responses to open-ended questions were categorized into major themes independently by two study investigators. Discrepancies in categorizing responses were resolved by consensus and involvement of a third investigator. Data analysis was performed using Microsoft Excel (Microsoft, Redmond, WA) or GraphPad (GraphPad Software, La Jolla, CA). A p-value of <0.05 was considered statistically significant.

 

Results

There were 32 pharmacy directors who completed the survey. The number of former residents who received the survey is unknown, however 97 completed the survey. Demographic and program characteristics are detailed in Table 1. The majority of program directors represented institutions offering PGY1 training (87.5%) and used a variety of projects types to fulfill the research requirement including: MUEs (65.6%), original research (81.3%), and quality improvement projects (78.1%). The minority of programs provided formal research training (41.9%) and most required projects to be in a format suitable for publication at the end of the residency (81.3%). In general, program directors were experienced (62.5% had been the program director for ≥5 years) and reported feeling comfortable or very comfortable (50.0%) with the research process.

 

 

The majority of residents had completed only a PGY1 residency (67.3%). Consistent with the director responses, the minority of residents reported receiving formal research training (44.9%) during their residency and that their project was required to be in a format suitable for publication by the end of the residency (83.7%). Almost half of resident respondents (46.9%) were not planning to publish their projects while 15.3% reported their project was either published or in press.

Overall, both the program directors and residents felt the projects were beneficial to the institution (73.3% vs. 64.5%, respectively, p=0.50) and provided an opportunity to develop adequate management and administrative skills (60.0% vs. 59.1%, respectively, p=0.93) (Table 2). Approximately half of program director and resident respondents felt resident projects adequately prepared them for conducting future research (46.7% vs. 50.5%, respectively, p=0.87). Only about a third of program director and resident respondents thought that conducting a resident project expanded career opportunities (33.3% vs. 35.5%, respectively, p=0.97).

 

 

The most commonly reported barriers to completing resident projects reported by directors and residents were developing a realistic timeline to complete the project in one year (40% vs. 45.2%, respectively, p=0.67) and working through the publication process (73.3% vs. 43.0%, respectively, p<0.01) (Table 3). However, 46.7% of program directors identified obtaining IRB or department approval as a significant barrier compared to only 22.6% of residents (p=0.02). There was also a significant difference between groups in the perceived barrier posed by both collecting (13.3% vs. 34.4%, p=0.02) and analyzing data (20% vs. 39.8%, p=0.04) with residents viewing these aspects as more challenging than the directors.

 

 

There were 29 program directors and 90 residents who identified at least one barrier in the open response portion of the survey (Table 4). Major themes that emerged included: time constraints, lack of resident motivation, IRB approval or compliance with other rules or regulations, limitations in study design or quality of the study, effective mentorship or program structural issues, resident knowledge gaps, and continuity issues following residency completion. Both program directors and residents reported that two of the top three barriers to publication were time constraints (58.6% vs. 62.2%, respectively) and limitations with the study design or quality of the study (59.6% vs. 56.6%, respectively). However, as the third barrier, program directors cited the theme of resident motivation (65.5%) while residents cited the theme of effective mentorship or issues related to the program structure (43.3%).

 

 

Discussion

While some literature currently exists assessing the perceived value of residency research projects⁹, limited information exists identifying barriers to conducting and publishing these projects. The barriers identified in our work are similar to those identified with medical resident research training and practicing pharmacists.^10,11 We hypothesized that perceived challenges may be different between residency program directors and former residents and indeed, we learned that while there were many similarities between the groups, there were important differences in key areas.

Both program directors and residents valued the pharmacy residency projects, felt they were beneficial to their institution, and helped residents develop project management and administrative skills. These findings support those of another study aimed at documenting the perceived value of resident projects.⁹ It is our belief that an important component of the ASHP resident project requirement is to increase the resident´s competence and comfort with the research process so that they will participate in practice-based research in some capacity throughout their careers. While our study did not evaluate research competence, only about half of directors and residents reported feeling comfortable with the research process. Similarly, about half of respondents agreed that the residency project adequately prepared them to conduct future research, also supporting similar finding from an earlier study.^12,13

Not surprisingly, common barriers identified by both program directors and residents involved generating quality projects that could be successfully completed within the time constraints of a one year residency. For a variety of reasons, this likely translates to low publication rates. Issues surrounding time constraints emerged as a recurring theme at all points of the research process including obtaining IRB approval, data collection and analysis, and manuscript preparation. Residents also noted that competing residency responsibilities further diminished their ability to work on research throughout the year. Time constraints may also result in many programs opting to engage residents in institution-specific small scale retrospective studies, such as MUEs or quality assurance projects, which often have limited external validity and are more difficult to publish. Most respondents indicated that projects were initiated between the second and third residency month. Recognizing that the first month of residency is extremely busy for both the program and resident, beginning projects as soon as possible during this time may help alleviate time constraints.

As discussed previously, while program directors and residents agreed on the first two self-identified barriers to project publication, they differed on the third. Whereas program directors frequently cited a theme relating to resident motivation, residents identified a theme relating to effective mentorship and program support. While reasons underlying this disconnect are not fully addressed by this study, we believe that our study results provide some insight and potential areas of opportunity. First, more than half of both program director and resident respondents reported a lack of a formal research training process and individuals from both groups identified resident knowledge gaps as a top barrier to publication. Studies have shown that objectively assessed research-related knowledge of residents is low after residency completion.^13,14 As a result, approaching the research requirement from the same structured process as many of the other residency components may help increase resident confidence and competence in the research process. Recognizing that this may not be feasible at all institutions, pairing residents with mentors experienced in the research process and motivated to produce high quality research may help achieve similar endpoints as it has with practitioners.¹¹ Additionally, complementing this mentorship with individuals with expertise in targeted areas, such as biostatistics or administrative data queries, may further facilitate movement through the various stages of the research process.

Second, while the vast majority of respondents stated a manuscript in a format suitable for publication was required for residency completion, very few residents published their work.^5-7 It has been previously remarked that residency research projects often fall short of the high standards set by peer-reviewed journals simply because projects are not given priority or designed for publication from the start.¹⁵ If publishing resident projects is a goal of residency programs, then programs might consider allowing multiple residents to collaborate on a single project¹⁶, offering projects that span multiple years, capitalizing on completed research with follow-up studies, or conducting a feasibility analysis prior to project initiation.

Both groups identified difficulties ensuring project continuity and publication following residency completion as another barrier. Once again, assigning an experienced researcher from the institution to each resident project may help in developing a post residency communication plan to ensure that projects do not languish. Additionally, setting clear expectations at the start of the residency about submitting and publishing the study results may help alleviate this barrier. Finally, a newly created journal dedicated to publishing pharmacy resident (past and present) projects offers another option for submission of these projects which may improve publication rates.¹⁷

Our study has limitations. We had a low response rate from program directors despite repeated reminders. While we still feel that the responses we received provided valuable insight in barriers to residency research training there is the possibility that these results are not reflective of the entire population of residents and program directors. Second, the survey responses were anonymous and thus we do not know specifically which programs responded which could have affected the results, as residency programs with a larger infrastructure may be more likely to have current contact information for past residents. This bias may be supported by the fact that 15.5% of residents stated that their residency project was either published or in press. This percentage is higher than that previously reported for WSC research projects of less than 10%^6,7, although consistent with a study of the SERC.⁵ Our inquiry about whether programs had a "formal research process" was not specific thus may have been interpreted differently among respondents.

 

Conclusions

In conclusion, it appears both residency program directors and residents agree that time constraints and low study quality are major factors limiting publication of resident research projects. However, program directors also felt that residents lacked the motivation to work through the publication process while residents felt poor mentorship and a lack of program support were larger factors. If programs wish to improve the publication rate of residency research projects, results of this study may provide some opportunities for improvement including formal research training that includes information on biostatistics, a plan for continued communication between resident and mentor upon residency completion, and an increase in the time allocated for research. Additionally, if developing pharmacy researchers is a goal of our profession, then perhaps strengthening the language in the accreditation standards to specifically say "research" may push programs to develop stronger original research projects.

 

Acknowledgements

The authors would like to thank Monica Evans who at the time of this study was a PharmD candidate at the University of Colorado Skaggs School of Pharmacy for helping to develop the surveys used for this study.

 

Conflict of interest

The authors have no conflicts of interest to disclose.

 

References

1. Murphy JE, Nappi JM, Bosso JA, Saseen JJ, Hemstreet BA, Halloran MA, Spinler SA, Welty TE, Dobesh PP, Chan LN, Garven CG, Grunwald PE, Kamper CA, Sanoski CA, Witkowski PL; American College of Clinical Pharmacy. American College of Clinical Pharmacy's vision of the future: postgraduate pharmacy residency training as a prerequisite for direct patient care practice. Pharmacotherapy. 2006;26(5):722-733.         [ Links ]

2. Haines ST. Making residency training an expectation for pharmacists in direct patient care roles. Am J Pharm Educ. 2007;71(4):71.         [ Links ]

3. American Society of Health-System Pharmacists. ASHP accreditation standard for postgraduate year one (PGY1) pharmacy residency programs. www.ashp.org/DocLibrary/Accreditation/ASD-PGY1-Standard.aspx (Accessed 2013 Feb 28).         [ Links ]

4. American Society of Health-System Pharmacists. ASHP accreditation standard for postgraduate year two (PGY2) pharmacy residency programs. www.ashp.org/DocLibrary/Accreditation/ASD-PGY2-Standard.aspx (Accessed 2013 Feb 28).         [ Links ]

5. McKelvey RP, Hatton RC, Kimberlin CA. Pharmacy resident project publication rates and study designs from 1981, 1991, and 2001. Am J Health Syst Pharm. 2010;67(10):830-836. doi: 10.2146/ajhp090090        [ Links ]

6. Olson KL, Holmes M, Dang C, Patel RJ, Witt DM. Publication rates of abstracts presented by pharmacy residents at the Western States Conference. Am J Health Syst Pharm. 2012;69(1):59-62. doi: 10.2146/ajhp110423        [ Links ]

7. O'Dell KM, Shah SA. Evaluation of pharmacy practice residents´ research abstracts and publication rate. J Am Pharm Assoc (2003). 2012;52(4):524-527. doi: 10.1331/JAPhA.2012.10224        [ Links ]

8. Hoffman JM, Thomley S, Vermeulen L, Smith KM. Pharmacy residency training in academic medical centers. Am J Health Syst Pharm. 2004;61(23):2528-2533.         [ Links ]

9. Murphy JE, Downhour N. Perceived value and outcomes of residency projects. Am J Health Syst Pharm. 2001;58(10):889-895.         [ Links ]

10. Rothberg MB. Overcoming the obstacles to research during residency: what does it take? JAMA. 2012;308(21):2191-2192. doi: 10.1001/jama.2012.14587        [ Links ]

11. Morris CT, Hatton RC, Kimberlin CL. Factors associated with the publication of scholarly articles by pharmacists. Am J Health Syst Pharm. 2011;68(17):1640-1645. doi: 10.2146/ajhp100660        [ Links ]

12. Streetman DS, McCreadie SR, McGregory M, Ellis JJ. Evaluation of clinical research knowledge and interest among pharmacy resident: survey design and validation. Am J Health Syst Pharm. 2006;63(23):2372-2377.         [ Links ]

13. Ellis JJ, McCreadie SR, McGregory M, Streetman DS. Effect of pharmacy practice residency training on residents´knowledge and interest in clinical research. Am J Health Syst Pharm. 2007;64(19):2055-2063.         [ Links ]

14. Bookstaver PB, Miller AD, Felder TM, Tice DL, Norris LB, Sutton SS. Assessing pharmacy residents´ knowledge of biostatistics and research study design. Ann Pharmacother. 2012;46(7-8):991-999. doi: 10.1345/aph.1Q772        [ Links ]

15. Hasegawa GR. Publication of residency projects: another perspective. Am J Health Syst Pharm. 2012;69(1):77-78. doi: 10.2146/ajhp110525        [ Links ]

16. Hellwig T, Laible B, Remund K, Fjeldheim C. Institutional collaboration on a residency research project. Am J Health Syst Pharm. 2013;70(6):484-485. doi: 10.2146/ajhp120619        [ Links ]

17. Journal of Health-System Pharmacy Residents. https://jhpr.org/about/ (accessed 2013 June 24).         [ Links ]

 

 

Received (first version): 03.04.13
Accepted: 23.07.13