INTRODUCTION

Ensuring that students successfully transition from didactic education to experiential education, and ultimately into autonomous practice, is a major objective of pharmacy education. To do so, pharmacy educators must link academic outcomes with activities commonly undertaken in practice. They must also ensure that students can undertake these activities with an appropriate (often a minimal) level of supervision when they graduate and pass their board exams.

Medical educators ten Cate and Scheele were the first to investigate the process of integrating practice abilities with academic competencies, which led to the creation of Entrustable Professional Activities (EPAs).¹,² EPAs are a collection of statements, each of which describes an activity that a pharmacist routinely undertakes with a reasonable degree of autonomy in practice.^{3 4 5}-⁶ EPAs are written in a manner that allows clinical preceptors to connect the competency statements currently used in didactic education to professional activities undertaken in clinical settings.⁵,⁶ When combined with a rating scale, EPAs with the rating scale represent a competency-based system used to evaluate health care professional training, including pharmacy education.^{3 4}-⁵ In doing so, EPAs attempt to provide a “common language” for both academic and experiential educators to assess the knowledge, skills, and abilities of their students.³,⁷

This manuscript focuses on the development and use of EPAs in pharmacy education. EPAs were introduced into academic pharmacy during the 2015-2016 academic year, when the American Association of Colleges of Pharmacy (AACP) Academic Affairs Committee released its six Core EPA domains for pharmacy: population health promoter, patient care provider, practice manager, information master, interprofessional team member, and self-developer.⁸ New Doctor of Pharmacy graduates should be able to perform each and every one of these activities with limited supervision (i.e., “Level 3” performance).⁵ Within pharmacy-based EPAs, this manuscript specifically focuses on the population health promoter (PHP) domain.

EPAs also provide a means to characterize the unique contributions of pharmacists to team-based patient care. This is especially true for the PHP domain, which is inherently interdisciplinary in nature.⁵,⁷ Pharmacist preceptors who undertake these activities provide pharmacy students with an opportunity to more fully grasp the impact of their practice on the health of patient populations as a whole, as well as a deeper understanding of how their practice integrates with, and supports, the work of other clinicians to ensure healthy populations.

While pharmacists practicing in any setting will undertake each EPA, it is also important to note that specific EPAs will be undertaken more or less frequently in specific pharmacy practice settings (institutional, community, ambulatory care, etc.).⁹ The more frequently pharmacists in a given practice setting undertake an EPA, the greater the opportunity for pharmacy students to develop greater levels of entrustability with that activity. Thus, before EPAs can be fully embedded in academic pharmacy, it is vital to assess the frequency with which pharmacists in certain practice settings actually perform specific EPAs in their daily practice. As a corollary, it is useful to determine whether the frequency of use varies by position and preceptor status, in addition to practice setting.

Previous studies have examined the perceived frequency of EPA use (both overall and across practice settings, by preceptor status, and by the type of position/responsibility) in the Patient Care Provider (PCP) and Practice Management (PM) domains.¹⁰,¹¹ Substantial differences were found in the perceived frequency of use across various practice settings. However, little is known about the perceived use of EPAs in the other four domains. This is especially problematic, since the continued evolution of the pharmacy profession is critically linked to its ability to develop an expanded role in population health, information management, and collaborations with other clinicians.

The primary objective of this manuscript is to conduct a pilot study evaluating the perceived frequency of use of EPAs in the PHP domain. As a secondary objective, this manuscript explores whether the perceived frequency of EPAs undertaken in the PHP domain varies across practice settings, across the seniority and responsibilities of the position, or by preceptor status.

METHODS

This study is a part of a larger initiative undertaken by the North Dakota Institute of Pharmaceutical Care to assess the frequency with which pharmacists in North Dakota perform EPAs in all six major domains. The methods utilized in this paper are described by other, previously published studies stemming from this initiative.¹⁰,¹¹ To implement the initiative, a survey was designed, using established criteria in the survey literature.^{12 13 14 15}-¹⁶ The target population consists of all pharmacists who are licensed and currently practicing as a pharmacist (as their primary location of practice) in the state of North Dakota. The research team used its knowledge of survey design, as well as the pharmacy services literature, to create an initial version of the survey.^{12 13 14 15}-¹⁶ Five pharmacists agreed to pilot test the survey and provide feedback for improvement, which was incorporated into the survey. Once those revisions were made, the survey was submitted to the NDSU Institutional Review Board, who approved the survey and its methods of administration in the fall of 2017.

The approved version of the survey was implemented using the Qualtrics Software System (http://www.qualtrics.com). The survey is comprised of four sections. The first section contains a cover letter (inclusive of explaining all rights and responsibilities of participants, in accordance with Institutional Review Board protocols) and an item designed to identify whether respondents meet the study’s inclusion criteria. The second section lists each core EPA, along with supporting tasks.¹⁰,¹¹ Based on the literature and the authors’ collective knowledge of pharmacy practice, the authors determined that the vast majority of practicing pharmacists would undertake activities in the practice manager EPA with much greater frequency than activities in other types of domains. This greater frequency of use allows respondents to report their perceptions of these tasks with an extremely high degree of consistency, accuracy and precision. Thus, for the practice manager EPA and all associated supporting tasks, respondents are asked to self-report the number of times during the last 30 days that they perform the task, using an 11 point response scale (0, 1, 2, 3, 4,..., 8, 9, 10 or more times). The authors were less certain that all pharmacists would be able to provide responses with the same degree of consistency, accuracy and precision for activities in other domains, including the PHP domain. Thus, for EPAs and supporting tasks in the other 5 domains (including the PHP domain), respondents are asked to self-report the number of times during the last 30 days that they perform the task, using a 6-point response scale (0, 1, 2, 3, 4, 5 or more times). In the third section, each of the six core EPAs was listed, and respondents were asked to rate (using a 1 “not useful” to 5 “very useful” response scale) the degree to which additional training on the core EPAs would be useful to further the pharmacist’s practice. The final section asks respondents to provide some basic demographics, including gender, age, name of the pharmacy school the respondent attended, year of graduation from a pharmacy program, highest degree earned, current primary practice setting, their primary position/level of responsibility, the location of their practice, and whether or not the respondent served as a preceptor for students attending the state’s lone school of pharmacy during the past year. The final section of the survey concluded with an open-ended question entitled, “Please add anything else you would like to tell us.”

An important consideration in the assessment of EPAs is the location of the practice setting.^{10 11}-¹²,¹⁴ Due to differences in patient populations, available staffing, financial resources, and local health services infrastructure, pharmacists practicing in rural areas may not undertake the same combination of EPAs (or with the same frequency) as those practicing in urban areas. North Dakota is an extremely sparsely populated state. More than half of its counties have population densities of less than six people per square mile and are designated by the U.S. Health Resources and Services Administration as “frontier counties”.¹⁰,¹¹ Concomitantly, four communities in the state (Fargo, Grand Forks, Bismarck, and Minot) have populations of 25,000 or more residents. The latter are typically categorized as “urban” communities, while all other communities are categorized as “rural”.¹⁰,¹¹

The North Dakota Board of Pharmacy provided a list of email or physical mailing addresses for all pharmacists currently licensed in the state. Using the physical mailing addresses, the investigators removed the names of pharmacists on the email list who reside well outside of the state’s borders (and cannot commute to North Dakota to practice), the research team was left with a set of 990 possible study participants. As an aside, more than half of North Dakota’s population resides in the Red River Valley along the Minnesota-North Dakota border. The 990 names likely include pharmacists who are dually licensed in North Dakota and Minnesota, but who primarily practice in Minnesota. Thus, this number likely overstates the eligible study population by a considerable margin.

The final, approved version of the survey was emailed to the 990 possible study participants in late September 2017. Consistent with the survey design literature, email reminders were sent 2, 4, and 6 weeks afterwards. Reminders were sent from the pharmacy program’s Senior Associate Dean in week 8, and from the Dean in week 10. A final email reminder was sent by the study’s principal investigator in the 12^th week of data collection. Data collection concluded in March 2018.

RESULTS

The survey was distributed to 990 individuals, of which 457 (or 46%) responded to at least one item in the survey. Of the 457 responses, another 102 were eliminated for failing to meet the study’s inclusion criterion of actively practicing as a pharmacist in North Dakota. Further analysis of these 102 responses indicates that a plurality were not practicing in North Dakota (n=33) or did not work for an employer that provides direct patient care (n=27). The remaining ineligible respondents were retired (n=28), unemployed (n=7) or employed in a non-pharmacy-related career (n=7). This yields 355 individuals who responded to at least one survey item in the demographic or PHP EPA sections of the survey. Unfortunately, a large number of individuals did not address one or more of these survey items, and many (if not the majority) of these respondents failed to answer many of the survey items.¹⁶ This result makes it infeasible to compare responders and non-responders. It is also unclear as to whether the additional information added by including responses from individuals who only partly completed the survey outweighs the inconsistency and increased information uncertainty introduced by allowing the sample size to vary from one survey item to the next. To avoid ad hoc or arbitrary decisions about whether and how to interpret missing values, a decision was made to include only those 121 individuals in the data analysis who provided a complete set of responses. Therefore, n=121 becomes the final sample size used in the analysis.

Descriptive statistics for all respondent demographics are presented in Table 1. Within the sample, 70% of respondents were female and 30% were male. The mean age of respondents was 43.2 years of age. Approximately 70% of respondents were less than 50 years of age. Approximately 50% of respondents held a Doctor of Pharmacy as their highest degree, while approximately 30% held a bachelor degree in pharmacy. A plurality of pharmacists (40%; n=49) reported working in an independent community setting, while another 26% (n=32) worked in a health system setting, and only 12% (n=15) reported working in a chain community pharmacy. These percentages are consistent with the pharmacy profession in North Dakota. More specifically, North Dakota has a legal requirement that (except for a few stores that were operating prior to the passage of the law) all community pharmacies in the state be majority owned by pharmacists licensed in the state.¹⁸ With regard to position, 38% (n=46) reported holding the position of pharmacy manager, 32% (n=39) reported working as staff pharmacists, 21% (n=26) as clinical pharmacists, and 8% (n=10) in other practice settings. Approximately 60% of pharmacists work in urban communities and 40% in rural communities. Fifty percent of all respondents serve as preceptors for pharmacy students attending the state’s lone school of pharmacy.

Descriptive statistics for all survey items in the PHP EPA domain (including supporting tasks) are presented in Table 2. The PHP EPA “Minimize adverse drug events and medication errors” was the highest reported EPA, and was performed an average of 3.4 times per month (SD=2.0). Within this EPA, the supporting task identified by respondents as the one most frequently undertaken was “Assist in the identification of underlying system-associated causes of errors” performed an average of 2.0 times per month (SD=1.9). The PHP EPA domain was undertaken the second most frequently was “Ensure that patients have been immunized against vaccine-preventable diseases”, which (at the sample mean) was undertaken 2.3 times per month (SD=2.3). Within this domain, “Determine whether a patient is eligible for and has received CDC-recommended immunizations” was the most frequently performed supporting task (mean=2.2 times per month, SD=2.3), followed by “Administer and document CDC-recommended immunizations to an adult patient” (mean=1.5 times per month, SD=2.2). With the exception of “Minimize adverse drug events and medication errors”), 60% of respondents reported performing a task 4 or fewer times in a 30-day window.

An analysis of responses by practice setting is presented in Table 3. The most commonly reported activity across practice settings was “Minimize adverse drug events and medication errors.” No significant differences exist across practice settings, suggesting that pharmacists in every setting perform this task with similar frequency. Significant differences were noted among practice settings for “Identify patients at risk for prevalent diseases in a population” (Kruskal-Wallis p=0.02; chi-square p=0.02), “Maximize the appropriate use of medications in a population” (Kruskal-Wallis p=0.02; chi-square p=0.02), and “Ensure that patients have been immunized against vaccine-preventable diseases” (Kruskal-Wallis p=0.01; chi-square p=0.01). When examining statistically significant differences across practice settings for these Core PHP EPAs, pharmacists working for chain pharmacies generally reported the highest mean values, while those working for independent community pharmacies reported and lowest means.

Table 4 contains an analysis of responses, disaggregated by position. As in Table 3, the item pharmacists reported the highest mean frequency was “Minimize adverse drug events and medication errors”. However, these differences were not statistically significant (Kruskal-Wallis p=0.43; chi-square p=0.29). The next most commonly undertaken item was “Maximize the appropriate use of medications in a population,” which also did not differ significantly by position type (Kruskal-Wallis p=0.19; chi-square p=0.06). Statistically significant differences by position did exist for the core EPA PHP domain “Identify patients at risk for prevalent diseases in a population” (Kruskal-Wallis p=0.02; chi-square p=0.17). In general, clinical pharmacists and pharmacy managers reported undertaking activities and tasks more frequently than staff pharmacists.

Table 5 reports the PHP domain by preceptor status (NDSU School of Pharmacy preceptor and non-preceptor). Statistically significant differences exist by preceptor status for the core EPA PHP domain “Identify patients at risk for prevalent diseases in a population” (Kruskal-Wallis p=0.01; chi-square p=0.04). For this item, preceptors self-reported undertaking the activity more frequently than non-preceptors. Mean values for “Minimize adverse drug events and medication errors” were very high, although significant differences did not exist across preceptors and non-preceptors (Kruskal-Wallis p=0.36; chi-square p=0.49). Concomitantly, two activity items within this domain, “Assist in the identification of underlying system-associated causes of errors,” (Kruskal-Wallis p=0.03; chi-square p=0.43) and “Report adverse drug events and medication errors to stakeholders,” (Kruskal-Wallis p=0.50; chi-square p=0.04) did elicit significant differences by preceptor status, with preceptors reporting higher mean values. For the remaining items, preceptor responses did not significantly differ from non-preceptors.

DISCUSSION

One of the most important contributions of Core EPAs is that they create a means to translate academic competencies to professional tasks. In that way, EPAs create a “shared language” inclusive of the evaluation of student performance - between academic and professional pharmacy.⁷,⁸ Unfortunately, a dearth of evidence exists demonstrating the frequency with which these EPAs (inclusive of supporting tasks) are undertaken in various areas of pharmacy practice.¹⁰,¹¹ The primary contribution of this pilot study was to help address this knowledge gap by assessing the self-reported frequency with which pharmacists practicing in North Dakota undertake Core EPAs in the PHP domain.

Overall, the primary conclusion of the study was that pharmacists perform activities and supporting tasks in the PHP domain, but do so inconsistently and not at a high level. For the four EPAs in the PHP domain, North Dakota pharmacists reported most frequently performing “Minimize adverse drug events and medication errors” (mean=3.4, SD=2.0), followed by “Ensure that patients have been immunized against vaccine-preventable diseases” (mean=2.3, SD 2.3), “Maximize the appropriate use of medications in a population” (mean=2.2, SD=2.3), and “Identify patients at risk for prevalent diseases in a population” (mean=1.3, SD=1.9).

While disappointing, the results of the study were not entirely surprising. The relatively low level of public health involvement is also supported by a 2016 study of North Dakota pharmacists along with those in Iowa and Manitoba, Canada.¹²,¹⁹ Most of the health prevention activities were medication-related and oriented towards individual patients (micro-level) rather than at a community (population-based) macro-level. Perhaps this was because most of the study participants worked in pharmacy settings that were: independents (n=49), hospital (n=32), chains (n=15), and other pharmacy positions (n=25), or they worked in pharmacy positions that were: managers (n=46), staff pharmacists (n=39), clinical pharmacists (n=26), and all other positions (n=10). In both cases, the primary activity is medication-related and most time is spent in dispensing activities, rather than population health promoter activities.²⁰ North Dakota has a unique state law requiring each community pharmacy to have a North Dakota-licensed pharmacist owning 51% or more of the pharmacy, thereby restricting the number of chain stores, and they are typically managed by the pharmacy owner who often serves as the primary pharmacist.¹⁸ Clinical pharmacists and pharmacy managers reported performing most of the tasks at a higher frequency than did the independent community pharmacists.

It is also important to note that pharmacists in North Dakota are becoming more involved in certain aspects of care that fall within the PHP domain. For example, one study found that targeted education improved the rate with which community pharmacists provide immunizations.²¹ Other initiatives are also underway in areas related to the PHP domain, including (but not limited to) opioid misuse.^{22 23}-²⁴ This suggests that it is also important to assess the self-reported frequency with which pharmacists undertake EPAs regularly over time. As targeted education initiatives are successfully undertaken, the frequency with which pharmacists undertake tasks in the PHP domain will also improve. This change, in turn, provides greater educational opportunities for pharmacy students to improve their proficiency in these areas of practice.

In our study, the PHP EPAs were also reported by preceptor status (NDSU School of Pharmacy preceptor and non-preceptor). In the first core PHP EPA “Minimize adverse drug events and medication errors,” one of two supportive tasks: “Assist in the identification of underlying system-associated causes of errors” was statistically significant, with NDSU preceptors performing those tasks more frequently, than did non-NDSU preceptors. In the second core PHP EPA “Identify patients at risk for prevalent diseases in a population,” the supportive task “Perform a screening assessment to identify patients at risk for prevalent diseases in a population (e.g., diabetes),” showed a statistically significant difference for preceptors more than non-preceptors. This difference may be explained by the availability of human resources. For example, pharmacists may be assigned students on a regular basis precisely because they can have greater time and other resources to reallocate their efforts away from “traditional” activities, and towards other aims, including activities in the PHP domain and precepting students. Precepting students may also create synergies which lead to pharmacists undertaking activities within the PHP domain. As an example, students often take the lead on performing the assessments while being supervised by a pharmacist. This supervisory role is less time consuming, and more efficient than performing the entire screening process by themselves. Consequently, this process often allows for additional overall services to be provided by the pharmacy.

It is also interesting to note that pharmacists who work in hospital and chain community pharmacies performed most of the core EPA PHP domain items and their supportive example tasks more frequently than did independent community pharmacists. This may be due to infrastructure advantages accruing to larger, more integrated, organizations. For example, hospital pharmacists may have access to the patient’s electronic medical record, which allows them greater opportunities to identify and provide services falling within the PHP domain. Chain community pharmacists often receive structured corporate support (directives and resources) to provide services including (but not limited to) immunization delivery and other preventive care programs (point-of-care testing, disease state management) that fall within the PHP domain. Independently owned pharmacies, which are the norm in North Dakota, typically do not have the same level of access to such infrastructure.

CONCLUSIONS

Pharmacists in North Dakota currently perform some population health promoter activities, but not at a consistent and high level. Most of the health prevention activities were medication-related and oriented towards individual patients (micro-level), rather than at a community (population-based) macro-level. With refinement, the EPAs in the PHP domain have potential as a means to assess outcomes in pharmacy education and practice. This study provides useful, but preliminary, basis for future research to further assess the use of EPAs in professional pharmacy practice (both more broadly across EPAs and deeply in the refinement of current EPAs), and how EPAs link to patient care outcomes.