INTRODUCTION

One of the most challenging areas of clinical education is ensuring that students successfully transition between didactic, experiential, and practice settings. Empirically characterizing and assessing these transitions are equally challenging endeavors. ten Cate and Scheele were among the first educators to investigate the process of creating and integrating practice proficiencies based on academic competencies.¹Their work, among those of other scholars, eventually led to the creation of Entrustable Professional Activities (EPAs).

EPAs, as used and assessed in pharmacy practice, are a competency system that is increasingly used in health professions education to evaluate healthcare professionals. The EPAs themselves are descriptions of work-related tasks performed by professionals practicing in a given field. Competence is assessed by asking a supervisor (usually a preceptor or professional supervisor) to rate the level of trust (using a standardized response scale) that the supervisor has in the student to complete the given task after acquiring the necessary knowledge, skills and abilities in a professional clinical education program. Each professional activity can be broken down into several specific sub-competencies.²New graduates (especially those in pharmacy practice) should be able to complete assigned tasks with reactive supervision (i.e., a Level 3 competency), while experienced clinicians should be able to complete the same task independently (i.e., Level 4 supervision).³EPAs are not meant to replace academic competencies, but rather to be used to incorporate them into clinical practice settings.⁴EPAs were first used by medical residencies to translate statements of competency into achievable tasks that can be observed.⁵In response to EPA formation in graduate medical education for family medicine and internal medicine, in 2014 the Association of American Medical Colleges created a list of 13 core EPAs that every medical resident, regardless of practice area, should be capable of performing their first day without supervision.⁶Since this publication, specific EPAs for other areas of physician practice, including (but not limited to) pediatrics, have been created.⁷,⁸

Pharmacy education is another field in which use of EPAs may be beneficial. Like medical education, pharmacy education is based upon a competency-centered framework. Many pharmacy preceptors find the competency statements currently being used (which primarily reflect the needs of academic programs) to be abstract and difficult to employ in clinical settings.⁵In theory, the incorporation of EPAs may provide a more effective and transparent framework for connecting didactic knowledge to experiential education. In doing so, EPAs facilitate assessment of students by preceptors, who tend to find EPAs conceptually easier to understand. They may also serve as an improved framework for students, who can observe these specific activities and emulate their preceptor as the preceptor displays these skills.³,⁹

EPAs domains were discussed for pharmacy curriculum by the 2015-2016 American Association of Colleges of Pharmacy (AACP) Academic Affairs Committee, which focused on converting the 2013 Center for Advancement of Pharmacy Education (CAPE) outcomes into distinct, measurable tasks.⁴,¹⁰The initial pharmacy EPAs required the integration of tasks and competencies within several different areas, such as knowledge, information collection and communication.⁴,¹⁰The 2015-2016 Academic Affairs Committee created six Core EPA domains for pharmacy: patient care provider, population health promoter, information master, practice manager, interprofessional team member, and self-developer.¹⁰As noted earlier, the Committee‘s expectation was that all new graduates, and subsequent new pharmacists, should be able to perform all of these activities with limited direction or feedback from supervisors (Level 3 performance).⁵Although these Core EPAs may be similar to those used in other health care professions, they have been adapted to emphasize the unique contributions of pharmacists to patient care, including interprofessional, team-based patient care.⁵,¹⁰,¹¹

EPAs are designed to standardize and strengthen the transition between educational training, residency, and practice.⁵However, as schools and colleges of pharmacy integrate EPAs into their curricula, it is important to determine the frequency in which pharmacists, and pharmacist preceptors, are completing each of the activities and tasks within each of the Core EPAs. It is thought that all the EPAs can be used in any pharmacy site (institutional, community, ambulatory care, etc.), but some will obviously better match with certain settings.¹²When looking specifically at the patient care provider EPAs, they appear to be geared toward an institution-based clinical experience, where patient records and other health care professionals are easily accessible.¹⁰Because of this, certain other settings, such as community pharmacies, may demonstrate these EPAs less often than other sites. If true, this could create difficulties for other types of practice settings to incorporate specific EPA activities and tasks into experiential education at those sites.

Prior to incorporating Core EPAs into pharmacy curricula and assessment practices, it is useful to assess how frequently pharmacists actually perform specific Core EPAs in their daily pharmacy practice. As a corollary, it is interesting to determine whether the frequency of use varies by practice setting, position, and preceptor status.

The study’s objective is to use survey methods to quantify the self-reported use of Core EPA activities and supporting tasks by pharmacists practicing in the state of North Dakota. Given the large number of Core EPAs, this study focuses only on EPA activities and supporting tasks categorized within the patient care provider domain. The survey responses were used to measure the extent to which pharmacists practicing in North Dakota perform them in various practice settings (hospitals, independent community pharmacies, chain community pharmacies, and other practice settings), by position (pharmacy manager, staff pharmacist, clinical pharmacist, all other roles), and by preceptor status (yes or no).

METHODS

The survey was designed using the North Dakota Pharmaceutical Care Survey as a template and employed standard survey research criteria.^{13 14 15}-¹⁶The survey incorporated the AACP’s six Core EPA domains, including those in the patient care provider domain, which is the focus of this manuscript. Within the patient care domain, there are five activity items, each of which is illustrated using multiple supportive example tasks. The five activities are: 1. Collect information to identify a patient’s medication-related problems and health-related needs (5 example tasks); 2. Analyze information to determine the effects of medication therapy, identify medication-related problems, and prioritize health-related needs (8 example tasks); 3. Establish patient-centered goals and create a care plan for a patient in collaboration with the patient, caregiver(s), and other health professionals that is evidence-based and cost-effective (6 example tasks); 4. Implement a care plan in collaboration with the patient, caregivers, and other health professionals (4 example tasks); and, 5. Follow-up and monitor a care plan (4 example tasks).

For each item and example tasks, the pharmacists were asked to respond to the question, “How many times in the past 30 days have you delivered the following services in your practice setting?” As noted previously, it is unclear exactly how frequently each of these tasks was conducted in practice, given the respondent’s practice setting. This makes the creation of an appropriate response scale challenging, since the scale must allow for meaningful responses, yet properly account for the wide range of possible responses that may be reported by pharmacists. The researchers determined that, if a pharmacist performs a given task 5 or more times per month, then over time in the position, the pharmacist would perform the task with sufficient frequency to develop and maintain a high level of expertise in completing that task. Within the context of this study, this means that responses of 5 times per month convey much the same information, as in the case where the same task was performed, for example, 20 times per month. Therefore, the researchers chose to use a 6-point response scale with the following options: 0, 1, 2, 3, 4, and 5 or more times per month. The survey also contained demographic items, including gender, age, highest level of pharmacy education, primary pharmacy practice setting, primary position, population of the city in which your pharmacy is located, and whether (during the past year) the pharmacy serves as a preceptor for NDSU pharmacy students.

The first survey draft was designed based on the training and experience of the research team. The instrument (inclusive of the choice of the response scale) was refined by comparing survey items against other surveys in the pharmacy literature and subsequently pilot testing it on five pharmacists. The survey was revised based on these processes.¹⁴,¹⁵All study procedures were reviewed and approved by the NDSU Institutional Review Board. Once approved, the research team obtained a list of pharmacist mailing and email addresses from the North Dakota Board of Pharmacy. From the mailing list, the investigators removed addresses outside the state of North Dakota, leaving 990 names in the final sample. The investigators used a modified Total Design Method, which has been used successfully in mail and internet surveys, to obtain a meaningful response rate.¹⁷,¹⁸Using the email address list, the survey was emailed to all pharmacists registered and living in North Dakota in September 2017. Survey reminders were emailed to respondents 2, 4, 6, 8, 10, and 12 weeks after the initial survey emailing.

Data analysis. The Qualtrics Survey Software (www.qualtrics.com) was used to compile the data collected from the on-line survey. Respondents were not identified in the analysis. Demographic variables were summarized using means and standard deviations for non-discrete variables, and proportions for discrete variables. One statistical concern associated with the survey is the interpretation of the EPA data collected using its response scale. We interpret the response scale as an interval scale, and its data as interval data with possible truncation.¹⁹More specifically, for those activities and supportive example tasks that most pharmacists frequently undertake, we expect a clustering of responses in the “5 or more times” category. This means that analyzing the responses with their exact responses, as provided, may generate descriptive statistics that understate the true mean and standard deviation for each survey item, especially if pharmacists complete each task many more than 5 times per month (instead of exactly 5 times per month). Concomitantly, for tasks that are performed infrequently, these descriptive statistics (inclusive of the response “5 or more times”) are likely to provide accurate and precise estimates of the true frequency with which pharmacists undertake these tasks.

Addressing this potential truncation requires a two-step analysis of responses that is both discrete (to distinguish the “5 or more times” from all other response options) and non-discrete (i.e., that uses the numerical value of the data, as they are reported). More specifically, we first report descriptive statistics (i.e., means and standard deviations) for each item over all possible responses. If the reported means are centered around the midpoint of the scale, then truncation is likely not a major concern (because there is little information loss that accrues from combining responses of 5 times per month with responses that exceed performing the task more than 5 times per month) and the descriptive statistics can be interpreted as reported. However, if the means are centered around the upper end of the scale, then truncation is likely to be an issue (because respondents are likely performing a task much more frequently than 5 times per month) and the basic descriptive statistics are not reliable. In such cases, an appropriate method of analysis is to change the nature of how the data are reported, by only summarizing information provided by the data that is not subject to possible truncation-related biases. More specifically, we report the proportion of responses for each item that report conducting the activity 5 or more times per month (versus the proportion of respondents that report conducting the activity less than 5 times), and the reader is encouraged to give greater weight to this measure, rather than the descriptive statistics for the entire scale.

When analyzing whether significant differences exist in the frequency of performing specific activities across different groups of pharmacy (i.e., by gender, practice settings, etc.) two different methods of analysis were used. Both methods operate under the null hypothesis of no relationship (or mean differences) between the frequency with which a task was completed and a specific pharmacist demographic. The chi-square test of homogeneity was used to assess possible differences in the frequency of respondents who undertook an activity 5 or more times, versus those who self-reported that they undertook the activity less than 5 times per month. The Kruskal-Wallis test (i.e., a nonparametric analog of analysis of variance assessing whether two or more groups of respondents exhibit similar distributions of responses) was used to assess whether specific pharmacist characteristics led to (self-reported) higher or lower frequencies of undertaking a specific task within a given 30 day window.¹⁹We note in passing that the Kruskal-Wallis test was used instead of alternative parametric tests because it may provide results that (in certain situations) may be less severely distorted should the data exhibit truncation. All tests were conducted using the IBM SPSS version 24 and utilized 5 percent significance levels.

RESULTS

A total of 990 pharmacists were initially deemed eligible to participate in the study, of which 457 individuals responded to the survey (46.1%). Of those who responded, 355 (78%) were licensed pharmacists practicing in North Dakota. The remaining [not employed in the state (n=33), not employed in a patient care setting (n=27), not in a related pharmacy related career (n=7), retired (n=28), and not currently employed (n=7)] were removed from the final sample. After further eliminating individuals who failed to respond to each of the patient care provider domain items, we obtained a sample of 107 responses (10.8%).

Table 1reports the demographic characteristics. Of the respondents, 67% were female. The average age was 43.5 years, with 68% of respondents reporting that they were less than 50 years of age. Most respondents had a Doctor of Pharmacy degree (67%) as their highest pharmacy degree, while 33% held a bachelor degree. The most common practice setting was independent community pharmacy (40%), followed by hospital (29%), all other practice setting (21%), and chain community pharmacy (10%). Thirty-six percent reported holding a pharmacy manager position, followed by staff pharmacists (32%) and clinical pharmacists (23%) positions. Sixty-eight percent of respondents served communities of 25,000 or more (e.g. urban) with the remaining 32% serving communities of 24,999 or fewer residents (e.g. rural). One-half of the respondents (50%) reported serving as an NDSU preceptor during the past year.

Table 2summarizes the descriptive statistics for the patient care provider EPA domain. The first patient care provider EPA activity “Collect information to identify a patient’s medication-related problems and health-related needs” was performed an average of 3.9 times per week (SD=1.8). Within this professional activity, the most commonly occurring EPA supporting example task was “Discuss a patient’s experience with medication” which was performed 3.8 times per week (SD=1.8). The next most commonly occurring EPA example task was “Determine a patient’s medication adherence” performed an average of 3.8 times per week (SD=1.9).

“Analyze information to determine the effects of medication therapy, identify medication-related problems, and prioritize health-related needs”, was the second highest reported patient care provider EPA activity, at an average of 3.8 times per week (SD=2.0). Within this professional activity, “Identify drug interactions” was the most frequently performed supportive example task (mean=4.5 times per week, SD=1.3), followed by “managed drug interactions” (3.6 times per week, SD=1.9). As expected, for the majority of items with high means, a majority of respondents reported that they perform a task close to or at 5 or more times in a 30-day window.

Differences attributable to practice setting are reported in Table 3Statistically significant differences were reported for “Analyze information to determine the effects of medication therapy, identify medication-related problems, and prioritize health-related needs,” (Kruskal-Wallis p<0.01; chi-square p<0.01). Significant differences were also noted across practice settings for “Establish patient-centered goals and create a care plan for a patient in collaboration with the patient, caregiver(s), and other health professionals that is evidence-based and cost-effective” (Kruskal-Wallis p=0.02; chi-square p<0.01); “Implement a care plan in collaboration with the patient, caregivers, and other health professionals” (Kruskal-Wallis p<0.01; chi-square p<0.01

EPA activities and supporting example tasks are disaggregated by pharmacist position, and are reported in Table 4Clinical pharmacists reported the highest levels followed by managers and staff pharmacists for all five activities: “Collect information to identify a patient’s medication-related problems and health-related needs;” “Analyze information to determine the effects of medication therapy, identify medication-related problems, and prioritize health-related needs;” “Establish patient-centered goals and create a care plan for a patient in collaboration with the patient, caregiver(s), and other health professionals that is evidence-based and cost-effective;” “Implement a care plan in collaboration with the patient, caregivers, and other health professionals;” and “Follow-up and monitor a care plan.”

With regard to EPAs and pharmacy position, there were significant differences in several supporting example tasks: “Use health records to determine a patient’s health-related needs relevant to setting of care and the purpose of the encounter” (Kruskal-Wallis p<0.01; chi-square p<0.01); “Interpret laboratory test results” (Kruskal-Wallis p<0.01; chi-square p<0.01); and “Perform a comprehensive medication review (CMR) for a patient” (Kruskal-Wallis p<0.01; chi-square p<0.01). Some supporting tasks were not significant, but were performed highly for all groups, including: “Identify drug interactions;” “Manage drug interactions;” and “Educate a patient regarding the appropriate use of a new medication, device to administer a medication, or self-monitoring test.”

Table 5reports the patient care provider domain activities and supporting example tasks, disaggregated by preceptor status, defined by NDSU preceptor during the past year versus not. For preceptors, the highly rated activities that was statistically significant were: “Analyze information to determine the effects of medication therapy, identify medication-related problems, and prioritize health-related needs” (Kruskal-Wallis p=0.03; chi-square p=0.03); followed by “Implement a care plan in collaboration with the patient, caregivers, and other health professionals” (Kruskal-Wallis p=0.02; chi-square p=0.06); and “Follow-up and monitor a care plan” (Kruskal-Wallis p=0.02; chi-square p=0.03). In each of these cases, preceptors reported that they performed the activities more frequently than non-preceptors.

In Table 5, four supportive example tasks were statistically significant. They include: “Collect a medication history from a patient or caregiver” (Kruskal-Wallis p=0.02; chi-square p=0.01); “Perform a comprehensive medication review (CMR) for a patient” (Kruskal-Wallis p<0.01; chi-square p=0.01); “Follow an evidence-based disease management protocol” (Kruskal-Wallis p=0.05; chi-square p=0.03); and “Write a note that documents the findings, recommendations, and plan from a patient encounter” (Kruskal-Wallis p<0.01; chi-square p=0.01)]. Four of the supportive tasks rated highly, but not statistically significant across preceptors and non-preceptors were: “Discuss a patient’s experience with medication;” “Determine a patient’s medication adherence;” “Identify drug interactions;” and “Manage drug interactions.” Another item ranked highly by both preceptors and non-preceptors was “Educate a patient regarding the appropriate use of a new medication, device to administer a medication, or self-monitoring test”.

DISCUSSION

Development of Core EPA domains in pharmacy education is an attempt to create a common vocabulary describing those activities and tasks that a new pharmacist should be able to undertake independently, as they are understood by practicing pharmacists.⁴,¹⁰,¹¹In doing so, EPAs may also serve as a practice performance measure.¹

There is a shortage of assessment data of the Core EPAs in the daily pharmacist practice in various practice settings. The frequency of performing Core EPAs by independent community pharmacy (40%) in North Dakota was followed by hospital (29%), all other settings (21%), and chain community pharmacy (10%). Pharmacists in “all other settings in pharmacy practice” (e.g., long-term care, ambulatory care such as health service, community health center, and medical homes) performed those EPA supportive example tasks more frequently than did hospital, independent, and chain community pharmacists. Perhaps pharmacists in these ‘other settings’ may have more opportunities for direct patient care versus traditional dispensing roles. Significant differences were noted among practice settings for 4 of 5 EPA activities: “Analyze information to determine the effects of medication therapy, identify medication-related problems, and prioritize health-related needs;” “Establish patient-centered goals and create a care plan for a patient in collaboration with the patient, caregiver(s), and other health professionals that is evidence-based and cost-effective;” “Implement a care plan in collaboration with the patient, caregivers, and other health professionals;” and “Follow-up and monitor a care plan.” These results are reasonable, since pharmacists practicing in “all other settings” have ample (and perhaps greater) opportunities to see complex patient cases, patients with lower health status, and patients experiencing acute symptoms that require pharmacist interventions that connect with these tasks.

North Dakota clinical pharmacists were more likely than other types of pharmacists (manager, staff) to report performing all five patient care provider EPA activities and supportive example tasks. For instance, “Collect information to identify a patient’s medication-related problems and health-related needs,” and within this activity, commonly occurring EPA supporting tasks were “Discuss a patient’s experience with medication,” and “Determine a patient’s medication adherence.” Similarly, clinical pharmacists in North Dakota were also more likely to report performing the other four activities than other types of pharmacists including: Activity #2 (4 examples), Activity #3 (6 examples), Activity #4 (1 example), and Activity #5 (4 examples), each of which was statistically significant. The findings show that clinical pharmacists were more likely than other pharmacists to perform the activities and supportive examples in the patient care domain. This is not surprising, since clinical pharmacists typically are pharmacy practice department faculty members who set-up their practice to role model advance practice skills and train pharmacy students in the provider care domain. Concomitantly, this finding also suggests that EPA activities and tasks in the patient care domain may be more difficult to characterize and assess in other practice settings.

In our study, the patient care provider EPA activities and supportive example tasks were disaggregated by preceptor status (NDSU preceptor and non-preceptor). Preceptors significantly performed the following more frequently than non-preceptors: “Analyze information to determine the effects of medication therapy, identify medication-related problems, and prioritize health-related needs.”; “Establish patient-centered goals and create a care plan for a patient in collaboration with the patient, caregiver(s), and other health professionals that is evidence-based and cost-effective;” “Implement a care plan in collaboration with the patient, caregivers, and other health professionals;” and “Follow-up and monitor a care plan.” Preceptors were more likely to role model, teach and practice these activities at their practice sites than pharmacists in other settings, and thus may be more aware than non-preceptors that they perform these activities on a daily basis.

In an attempt to bridge the gap between the didactic and experiential curricula, and between specific AACP domains and EPAs, the EPAs have been mapped at this school of pharmacy to ability-based outcomes (representing AACP domains and CAPE outcomes). The school’s Introductory Pharmacy Practice Experience (IPPE) and Advanced Pharmacy Practice Experience (APPE) programs integrated EPAs into training and assessment of pharmacy students in experiential education. In summary, this study’s survey appraisal represents an early assessment of this process (pilot test). As this integration process continues, the occurrence of these activities and supportive examples are expected to increase.

CONCLUSIONS

This study provides empirical evidence (albeit partial) suggesting that EPAs have potential as a useful means to assess outcomes in pharmacy education and practice. Our study sets the stage for future work that further refines and assesses core EPA activities and supportive example tasks to measure the impact of how this process relates to outcomes of care.