Contribution to scientific literature

The objective of this original article was to reduce fatigue alert in our Assisted Electronic Prescription System, through the optimization in the number and quality of alerts, and the standardization of the pharmacist action when faced with the different types of alerts.

The fatigue alert we have detected in our daily practice is something frequent in Assisted Electronic Prescription (AEP) alert systems. It has been confirmed that many deficiencies are still identified, in spite of the wide experience and the literature available about the implementation of these systems. Moreover, there is a limited number of studies about their efficacy in terms of the relevance and quality of the information offered, as well as about the response of users to this information, and all this can compromise the efficacy and safety of the system.

We believe that the reason for both the originality and the relevance of these outcomes is the methodology followed, Lean Six Sigma, which presents the characteristic of being supported by a work team, that is to say, it requires consensus by users; and most of all, improvements are introduced according to the “Client Voice” or user opinion. Regarding this, we consider that there was a enough level of participation (approximately 10% of hospital physicians).

This evaluation of the alert system by a high number of users, together with the analysis of the quality of the most frequent types of alert and the improvements implemented, will gradually generate valuable information about the real utility of alert systems in AEP programs.

Introduction

Assisted Electronic Prescription Systems (AEPS) represent, in the setting of safety improvement for patients, an important support tool for making clinical decisions at the time of prescription¹.

These systems have had a significant development in recent years; therefore, besides a wide range of clinical information, the majority of them include a potent system of safety alerts. However, there are few studies on their efficacy, and their clinical benefits for patients have not always been demonstrated^2,3,4,5.

It has been described that the alerts generated by an AEP system are ignored in up to 49-96% of cases³. This shows a problem in user adherence to recommendations; one of the main causes for this is the high number of alerts generated (between 34 and 90% of prescriptions); moreover, these are not always relevant, specific, and easy to manage. All this will lead to the development of “fatigue”⁶.

Peterson and Bates⁷ describe “alert fatigue” as “the state of mind resulting from receiving too many alerts, which are time-consuming and require too much mental energy, and that can be the cause for some clinically relevant alerts to go unnoticed among other non-relevant alerts”. That is to say, there is a desensitization of the user, who ignores the alerts that are not important as well as those that are important, thus compromising the objective of these alerts, which is patient safety.

Our AEP system features various information and safety levels. Firstly, it provides basic information that will make prescription easier. Secondly, it conducts a basic check of the prescription, according to a knowledge base of reference (defined regimens, maximum dosing, maximum number of treatment days, interactions, contraindications and duplications). And finally, it includes an advance check that integrates relevant patient data, such as history of allergies and certain clinical conditions, even decision algorithms, such as the renal impairment module for dose adjustment in those medications that require it, and dosing based on body weight and/or age for paediatric patients. All this will generate alerts that appear interactively and/or as a report at the end of the prescription (this option can be set up by each user based on their preferences and experience). But the prescription is never blocked, and there is no obligation to justify the decision to ignore the alert; and therefore, there is no feedback that would be required for updating the system¹.

Despite this, and the fact that there has been a selection of clinically significant alerts, and these have been customized according to the clinical situation of the patient and adverse event severity, the presence of alert fatigue has been detected in our daily practice. Therefore, it is necessary to improve the efficiency of the alert system, reducing the workload but without any negative impact on patient safety.

These improvements require consensus by users and clinical committee; they must be introduced cautiously and their results must be assessed. Therefore, this project has been developed following the Lean Six Sigma (LSS) methodology; its use is not widespread in the healthcare setting, but it has already provided some successful experiences⁸. Indeed, this is a scientific methodology for continuous quality improvement, supported by a work team, and understanding client needs. The key idea is to apply Lean tools to eliminate any unnecessary stages without any value (waste), and reduce the cycle times (efficiency), as well as the Six Sigma Techniques in order to reduce variability and deficiencies (effectiveness and safety)⁹.

The objective of the present article is to optimize the number and quality of alerts in our AEP program, to standardize the action by pharmacists when faced with different types of alerts, and to increase user satisfaction.

Methods

Population and sample

The study included the alerts generated during two trimesters (before and after the intervention), obtained from the AEP system. For the qualitative analysis of the project, 496 prescriptions in total were reviewed.

Schedule of the DMAIC cycle

The LSS project was conducted between May, 2014 and May, 2015. It was structured on the DMAIC cycle stages.

In the Define Stage (May-June, 2014), the problem, objective, work team and project schedule were defined. The Work Team was formed by five users of the AEP system (4 pharmacists and 1 physician), an external observer physician, a pharmacist for methodological support, and the pharmacist managing the AEP system database, as group liaison.

According to the LSS methodology, the first step is learning the “Client Voice” about the alert system in the program. For this aim, a survey was designed with six questions about the quantity and quality of alerts, as well as their attitude towards them; this survey was distributed to all physician and pharmacist users (specialists and residents). The survey was completely anonymous, although it included data about the clinical department and the professional category.

Based on the analysis of Client Voice, the “critical characteristics” of the project were determined (LSS methodology), and those quantitative and qualitative indicators to be measured were defined (Table 1).

Table 1 Critical characteristics derived from Client Voice and quantitative and qualitative indicators 

In the “Measure, Analyze” Stages (July-September, 2014), those alerts generated during the first trimester of 2014 were quantified, as well as the number of hospitalized patients and the number of lines of prescription modified during this period.

In order to measure the qualitative indicators, the most frequent types of alert were analyzed, as well as the molecules responsible for over 50% of each type of alert. A sample of the prescriptions generating these alerts was analyzed, from the perspective of alert relevance (whether adequate or not), and of the clinical action taken for these alerts (acceptance or not) (Figure 1).

Figure 1 Alert Classification and Definition of Quality Indicators 

The pharmacist action for the alerts most valued by physicians was assessed through a second survey.

In the “Improve” Stage (October-December 2014), it was considered to eliminate and modify those alerts that were no valuable, according to the predefined criteria for a good alert system^3,6,10. For this aim, the improvements to be implemented were prioritized, and the work team reached a consensus. The initial objective established was the reduction by half of the alert load.

With the aim to reduce the variability in pharmacist action for different types of alert, a series of recommendations were made to the Pharmacy Team.

In the “Control” Stage (January-May, 2015), the indicators were reassessed, based on data extracted from the AEP system for the first trimester in 2015, once the improvements had been implemented.

There was an estimation of the benefit in the reduction of the number of alerts, based on the assumption that the time required to read an alert is 1.7 seconds.

Statistical Analysis

The two-proportion z-test.was used for the statistical analysis of the outcomes. The predetermined level of significance was 5%. The analysis was conducted with the statistical analysis G-Stat 2.0 program.

Results

Survey (Client Voice)

In total, 136 surveys were collected from 24 clinical departments, both medical and surgical. A 69% of the participants were specialists, and 31% were residents. The results of the survey are shown in Table 2.

Table 2 Results of the survey (Client Voice) on the alert system (n = 136) 

Ẋ = arithmetic mean standard deviation

These results were translated into the critical characteristics of the project, and an indicator was defined for each one, with its criterion and measurement (Table 1).

Initial situation

During the first trimester in 2014, there were 47,994 different alerts; their rate of distribution by type of alert is shown in figure 2.

Figure 2 Percentage distribution of the type of alerts at the initial situation 

The results of the quantitative indicators appear in Table 3.

Table 3 Quantitative Indicators 

For estimating the quality indicators, there was a selection of 71 out of 971 molecules (7.3%), responsible for over half of the alerts generated for each type of alert (58%). After the assessment of each type of alert, these could be classified into two groups:

Alerts caused by software limitations: therapeutic duplication, dispensing conditions, and allergies.

Alerts that represented a problem in the quality of the alert or the action by the user, such as those for renal impairment, geriatrics, maximum doses, or interactions. For their analysis, 269 prescriptions were reviewed, and the predefined quality indicators were estimated (Table 4).

Table 4 Quality Indicators 

*Values weighted per sample size (n)

Table 5 shows the results of the survey on pharmacist action for those alerts valued as the most relevant by physicians; this survey was distributed among 19 Pharmacists (5 residents, 14 specialists).

Table 5 Survey on Pharmacist action for the alerts most valued by physicians (n = 19) 

Improvement Actions

A potential improvement was suggested for each type of alert and molecule, according to the alert generated and its quality. The improvements to be implemented were prioritized according to their feasibility and efficiency, there was a risk evaluation, and consensus was reached in the work team about the measures to be taken. Sixty (60) fields were modified in the alert systems, corresponding to 32 molecules (Table 6).

Table 6 Improvement actions for the alert system 

In order to reduce the variability of Pharmacist action when faced with an alert, the work team agreed by consensus on a series of recommendations, which were distributed among Pharmacists (Table 6).

End of the project

The improvements and project results were reviewed and discussed by the work team, and then communicated to healthcare professionals through the hospital intranet by computer graphics (Figure 3).

Figure 3 Computer graphics published in the hospital intranet to communicate the project outcomes 

Discussion

The ideal AEP system would provide information to the user that is perceived as very useful, and achieve the adequate balance between the number of alters and the guarantee of patient safety. Our alert system already features the essential alert categories described in literature, and those characteristics defining a good alert system, such as specificity, sensitivity, clear and justified information, stating severity and the action required, and no interruption of the work flow (only the most important alerts appear in an interactive format).¹¹ However, there is room for improvement. And this article is an approach to real practice, based on user opinion about the alert system, in terms of quantity and quality, in order to suggest measures to reduce alert fatigue.

According to the survey, the majority of prescribers (93%) claim that they read the alerts, but only 30% declare that they always follow their recommendations, which shows system alert fatigue. Prescribing physicians become sceptic regarding the validity of the system, trusting their own knowledge or assumptions, and/or underestimating the potential adverse effects of medications¹⁰. In fact, very few users (6%) acknowledge that they have detected some negative consequence by not following the recommendations. This coincides with the results of previous studies, where adverse effects have been detected only for 2.3 to 6% of those alerts ignored³.

Regarding the total number of alerts, even though it was not considered excessive by users, it has been reduced by 28%. Therefore, from visualizing an alert for each 3.6 lines modified, there is now an alert per each 5.1 lines modified. And all this without any negative impact on patient safety, because there is still a 100% proportion of alerts ignored with justification, and only those alerts ignored without justification represent a threat from the point of view of safety.

The most valued alert is that regarding drug allergies; and practically all participants (89%) answers that they would like the line of treatment to be blocked when this type of alert appears. In our case, even though there is high sensitivity, it only generates 2% of the total number of alerts, but still presents false positive results, mainly due to an incorrect record of the history of patient allergies (real allergy vs. lack of tolerance) and/or when the alert is based upon potential cross-sensitivities^12,13. The improvement consisted in standardizing pharmacist action at the level of allergy coding and intervention to be conducted.

The renal impairment alert is the third most valued alert, and responsible for 25% of those alerts recorded. The information available has been reviewed and updated, in order to improve its perceived utility.

Four molecules were practically responsible for geriatric alerts, mainly regarding a gradual increase in dose and monitoring in geriatric patients when, in the majority of cases, these medications were already part of their chronic treatment; therefore, it was decided to eliminate these recommendations.

Regarding maximum doses, low adherence has been detected, which can only be corrected through pharmacist intervention.

In terms of interactions, it is known that the sensitivity and specificity of interaction alerts have a high impact on the quality and number of alerts presented. It is recommended to include only those which are clinically relevant and require some action¹⁴. The module of interactions has been reviewed and updated, and pharmacist action has been improved. Algorithms must still be introduced that combine the prescription of one or more drugs with lab test data and their adequate range; this would increase the specificity of these alerts, because one of the most frequent reasons to ignore them is the intent to monitor the patient^13,15,16. The incorporation of these highly specific algorithms would also allow to alert about other contraindications in general, which is information highly valued by physicians, according to this survey.

All this has achieved a 25% reduction in false positive results; that is to say, there has been an increase in the utility of the alert systems (PPV). But alert fatigue is a complex situation, associated with a human component which determines that the user will read the alert and value its level of utility at the time of making a clinical decision. Therefore, the real impact of reducing the volume of alerts is still to be determined^16,17. And even though the number of alerts has been reduced, no improvement has been achieved regarding the level of adherence by the user, who continues eliminating them instinctively. In order to improve this adherence, it will probably be necessary to inform users about these improvements; and this has been done by publishing computer graphics in the hospital intranet, and allowing for a period of adaptation.

Regarding the personal alert setup (question F), very few medical users remember that they can change their profile (after their initial training). Even though this is one of the factors most frequently mentioned in literature as an example of how to optimize the alert systems, in the study by Jung et al. it appears as the aspect least valued by physicians¹⁸. And according to the opinion of users in our centre, this is not an option that they consider essential for the improvement of the system.

One step further in order to improve process efficiency has consisted in reviewing the variability in pharmacist action when faced with the alerts. Through a training session we have standardized our action, defining how, when, who, and in which circumstances there must be an intervention.

An important aspect in the future will be to establish the relationship between making clinical decisions by alert and the quality of patient care or health outcomes. In fact, an evolution in AEP system information towards health outcomes is already being demanded^14,17.

As a conclusion, there has been a quantitative and qualitative improvement in the alert system, following the LSS methodology. However, continuous maintenance will be required, with iterative refining and function monitoring, in order to ensure an optimal function of these tools for supporting clinical decision, as well as trust by the user and their ability to react when faced with the information offered,

The authors declare that there is no conflict of interests, and that there has been no funding whatsoever for this project.