Purpose A software-based tool to help prioritize inpatients for adverse drug event (ADE) prevention initiatives is described.
Summary The clinical pharmacy department of a New Zealand hospital developed the Assessment of Risk Tool (ART), an application for monitoring prespecified clinical “flags” (some derived from the Institute for Healthcare Improvement’s ADE trigger tool) for high-risk medication use and other ADE risk factors. The ART permits ADE risk assessment in virtual real time (i.e., medication-use data and other clinical information are updated multiple times daily). Each of the 38 flags captured by the ART is assigned a weighted score; the item scores are summed to provide a total ART score indicating low, medium, or high ADE risk, and patients are prioritized by the ART score for pharmacist interventions such as clinical review and discharge coordination. In the first 18 months after ART implementation, the average number of patients receiving medication reconciliation each month increased from 280 to 500. During one 8-month period, 765 high-risk patients were prioritized for discharge services and 526 medication errors (MEs) were prevented, including 174 errors deemed to pose a threat of moderate-to-major patient harm. The tool has been well received by clinicians and has generated interest among other New Zealand hospitals.
Conclusion By facilitating the identification and monitoring of patients at high risk for MEs and ADEs, the ART has enabled one hospital’s clinical pharmacists to conduct interventions such as medication reconciliation and clinical review in a more timely and targeted manner.
Adverse drug events (ADEs) affect an unacceptable number of hospitalized patients each year, with some resulting in permanent disability or even death. The 2011–12 version of the New Zealand “Serious and Sentinel Events Report” identified medication errors (MEs) as comprising the third largest category of events in the country’s hospitals, occurring in an estimated 5% of all hospital admissions.1 In the United States, it was estimated in a meta-analysis published in 1998 that ADEs (either as a cause of or consequent to hospital admission) ranked among the fourth to sixth leading causes of death among hospitalized patients.2
Through specific interventions, hospital pharmacists are ideally positioned to minimize ADEs. Pharmacist interventions that have been proven to reduce ADEs and optimize patient therapy include medication reconciliation, daily patient medication review, pharmacist participation on rounds, education on medications, and discharge coordination.3–6 Wider use of the medication reconciliation process, which takes a systematic approach to obtaining accurate medication histories by identifying unintentional discrepancies, has been identified as a National Patient Safety Goal by the Joint Commission.3,7 Numerous studies have demonstrated that medication reconciliation reduces MEs and ADEs at admission, at transfers of care between inpatient care settings, and at discharge.4,8
However, with limited pharmacist resources and growing numbers of hospital admissions, many hospitals find it challenging to undertake medication reconciliation for all patients across the continuum of care.9 Some experts have suggested targeting patients thought to be at high risk for experiencing ADEs and giving them priority for medication reconciliation and subsequent interventions.9 One approach involves risk stratification, in which patients are stratified according to their risk of an event, using prospectively gathered information. This has been an effective clinical approach for reducing event rates in other areas of health care—for example, decreasing cardiac events in patients undergoing noncardiac surgery.10 By prioritizing services in this way, it may be more feasible to provide early interventions to targeted patients.9 The challenge for the clinical pharmacist lies in accurately identifying high-risk patients.
This article discusses the development of the Assessment of Risk Tool (ART), an electronic patient prioritization tool, by the clinical pharmacy department at Middlemore Hospital in Auckland, New Zealand. The ART was designed to improve pharmacist resource management so that patients at high risk for medication-related harm receive early interventions, using available resources, with the aim of reducing MEs and ADEs.
Risk factors for ADEs
The literature reveals numerous factors thought to be associated with an increased risk of ADEs in hospitalized patients. These factors can be divided into four categories, as described below.
High-risk patient groups
The literature suggests that older adults on multiple medications are at greater risk for ADEs, particularly patients who are 70 years of age or older and have dementia, multiple chronic medical problems, or renal insufficiency or had a recent hospitalization.11–16
Polypharmacy has also been linked with an increased risk of drug-related problems and ADEs.17 In the Safer Systems Saving Lives project (a national collaborative initiated by the Australian Council for Safety and Quality in Health Care), which categorizes patients into three ADE risk levels (high, medium, and low), concurrent use of five or more medications is considered a high-risk factor.18 Another clinical pharmacy risk assessment tool, developed by the Canadian health services delivery organization Northern Health attaches a weighing score to patient risk; using this tool, patients on eight or more medications receive a high-risk score.19
High-risk hospital settings
Research has found that ADEs are more common in the intensive care unit (ICU) compared with general care units.28 Patients discharged from acute care hospitals are also at high risk for unintentional discontinuation of medications, with an even greater risk among those with an ICU stay.29
High-risk social settings
Social factors associated with an increased risk of ADEs have been identified. In Australia, ADE risk is higher among patients with limited English comprehension and those from economically disadvantaged backgrounds, as well as patients who have frequent hospital admissions and those who use multiple prescribers and pharmacies.13
Middlemore Hospital is a public hospital operated by the Counties Manukau District Health Board (CMDHB), Auckland, New Zealand. The hospital caters to an ethnically diverse population largely consisting of people of Maori and Pacific descent, with a high proportion of the population living in the most socioeconomically deprived communities. The rapidly aging population and the health impacts of socioeconomic deprivation pose health care challenges. The hospital has approximately 900 beds and a monthly average of 6000 patient admissions. In 2010, electronic medication reconciliation, performed at admission, was introduced. As electronic medication reconciliation was implemented across the various specialties, it became evident that due to large patient numbers and high patient turnover, medication reconciliation could not be performed for all patients admitted. Therefore, clinical pharmacists had to manually prioritize their list of patients for medication reconciliation and clinical review. Manual prioritization of patients was based on the individual pharmacist’s clinical judgment and experience and was a subjective and time-consuming process that typically took 1.0–1.5 hours per 8-hour workday.
At the time development of the ART was initiated, there were approximately 21 clinical pharmacist full-time equivalents (FTEs) at the hospital. The staff was deployed using a centralized model, with 1 pharmacist assigned to multiple wards and serving approximately 30–60 patients each day. Thus, less-intensive patient monitoring and intervention activities such as medication chart screening were used for the majority of patients; few patients received medication reconciliation and pharmacist review, and the numbers of patients receiving these interventions were not routinely recorded. Discussions with members of other hospital pharmacy departments in New Zealand revealed similar problems with workloads and patient prioritization.
Therefore, after the introduction of electronic medication reconciliation in 2010, senior clinical pharmacists, in collaboration with the hospital’s Centre for Quality Improvement and with assistance from information technology (IT) staff, set out to develop an automated prioritization tool to overcome the problems of manual patient prioritization.
Development of the ART
In an initial attempt at electronic patient prioritization, a spreadsheet report was generated with IT staff assistance using archived data in order to capture some of the key risk factors for MEs and ADEs (i.e., age, ethnicity, number of medicines, recent hospital admission, frequent presenter, high-risk transfer, high-risk specialty patient, multiple outpatient visits). This was a generic report simply identifying all patients at risk within the hospital.
Because the report was e-mailed to pharmacists from a computer in the pharmacy department each morning, the process was dependent on the efficiency of the central communication link.
Despite its value in guiding pharmacist workflow, the initially developed electronic reporting method had a number of limitations. Data sources used to generate the report were limited and reflected the previous day’s patient care activity; therefore, the information often had limited clinical relevance. Due to the large data set and complex queries, reporting performance and reliability were ongoing issues. The reports provided only a point-in-time snapshot of a patient’s perceived risk, and data could not easily be aggregated to view changes in a patient’s risk profile over time during an inpatient stay.
Addressing the deficiencies of the initial reporting method required migration to a more robust platform and access to multiple electronic data sources in order to capture and collate additional and more up-to-date risk information. This led to the development of a virtual real-time electronic tool that could be readily accessed by the clinical pharmacist—at any time, from any location in the hospital—to identify patients according to their level of risk for MEs and ADEs.
The ART, a predictive risk-profiling tool, was designed as an application within the Middlemore Hospital information system (Concerto Medical Applications Portal, Orion Health, Auckland, New Zealand). The tool enabled a systematic and transparent approach to patient prioritization based on specific clinical criteria known to be associated with increased risks of ADEs and MEs. The ART was designed with a total of 38 risk “flags,” divided into five groups, tailored to the hospital’s patient population. The flags were reviewed by senior pharmacists at Middlemore Hospital, and appropriate scores were derived for each flag through a group consensus process. Each flag was assigned a score ranging from 1 to 10, with 10 being the highest possible score for an individual flag. In using the ART, the scores for all triggered flags are summed to provide a patient’s total risk score. Based on the total ART score, patients are categorized as being at high, medium, or low risk for MEs and ADEs.
Subsequently, appropriate electronic data sources available within the hospital were identified to extract the necessary information. These sources included the hospital’s patient information management system (iPM-iSoft Patient Management, Computer Sciences Corporation, Falls Church, VA) for patient demographic data; Soprano Medical Templates modules (Orion Health) for clinical documents such as electronic discharge summaries and medication reconciliation documents; MediConnect Reporter (PharmSoft Limited, Auckland, New Zealand), for managing medication-use information obtained from transaction data from automated dispensing cabinets; and Éclair Clinical Information System (Sysmex New Zealand Ltd., Auckland, New Zealand), for laboratory data. Data were extracted from these sources three times daily, with updates of patient risk scores at 06:00, 10:00, and 13:00 hours.
The ART model design, business analysis, and testing were provided by the pharmacy team and were estimated to have taken approximately 320 hours. The time required for consensus-building meetings of pharmacy team members, who reviewed the risk flags and allocated scores for each, was estimated at 160 hours. The pharmacy team worked closely with IT staff within the organization so that the cost of development of the tool was minimal. The programming of the tool, performed by inhouse IT experts, was estimated to have taken approximately 2000 hours.
Assessment of risk flags
Tables 1–5 describe the 38 ART flags. The first group (Table 1) is the ART Patient Profile flag group. The data for flags 1–3 are extracted from the patient information system. Flags 4 and 5 (poor medication compliance and English difficulty, respectively) are triggered only after an electronic medication reconciliation document is completed by a pharmacist. Although these two flags carry the lowest risk score (i.e., 1) on the basis that a pharmacist has seen the patient and completed medication reconciliation, there is still value in flagging the patients for intervention prior to discharge. The data for flags 4 and 5 are extracted from Soprano Medical Templates.
The ART Patient Encounter flags (Table 2) are derived from the patient information system. These flags aim to identify medically complex patients, who often receive a number of high-risk medicines and are thus at an increased risk of MEs and ADEs, including patients not staying in their service-specific wards.
The ART Clinical Profile flag group (Table 3) identifies patients with chronic illnesses who are likely to be on complex medication regimens and require ongoing review of their medications. Middlemore Hospital, as part of an initiative by CMDHB, has a Chronic Care Management (CCM) Programme that entitles eligible patients to five free visits to their general practitioner each year. These patients include those with cardiovascular disease, stroke, respiratory disease, depression, or diabetes. Patient enrollment and clinical details are extracted from the CCM database.
The majority of wards at Middlemore Hospital use Pyxis MedStation automated dispensing systems (CareFusion Corporation, San Diego, CA) for access to patient medications. Table 4 displays the ART High-Risk Medications flag group. The information for flags in this group (Pyxis machine–generated ward dispensing data) and extracted from MediConnect. Flag 8 was created to identify cases in which a patient’s medication profile has not been recently updated (i.e., a pharmacist has not reviewed and updated the patient’s medication profile on the Pyxis system within 72 hours).
The ART Laboratory Values group (Table 5) is the largest, with 13 flags. These were derived from some of the triggers found in the Institute for Healthcare Improvement (IHI) ADE trigger tool30 and were later validated through consensus meetings with a national panel of senior pharmacists (appendix). Data for these flags are extracted from an Éclair-based web service (Sysmex New Zealand Ltd.), which provides all laboratory test results for patients admitted to Middlemore Hospital.
After the development of the risk flags and their associated scores, virtual patient testing was undertaken in a test environment to ensure that the flags were correctly triggered for patients in the various risk groups. Further testing was conducted after ART implementation to confirm that patients were flagged correctly during actual use of the ART.
Risk score ranges
For a period of approximately two weeks during the test period, trends in patient risk scores were monitored daily to help determine risk score ranges. Based on clinical assessments of risk associated with the cumulative ART scores (maximum possible total score, 221) and pharmacy department staffing constraints, it was decided that the top 10% of adult admissions by total ART score (i.e., those with scores above 22) would be flagged as being at high risk for ADEs; those in the next highest 15th percentile (total ART scores of 11–22) were categorized as medium-risk patients, and the remaining 75% of patients (total ART scores of 10 or under) were categorized as being at low risk for ADEs and MEs.
Acute psychiatry inpatients are excluded from ART assessments because they are deemed to be at a relatively lower risk for ADEs, as the hospital provides specialist psychiatry services and patients are under the direct supervision of the appropriate care team. The ADE risk of community-dwelling psychiatric patients admitted to the hospital but not under the direct care of the psychiatric services is deemed to be higher (Table 2, flag 3).
The ART application home screen, as seen in Figure 1, uses a color-coding system (high-risk scores highlighted in red, medium-risk scores in yellow, and low-risk scores in green) and lists patients by ward, clinician, or specialty (as chosen by the clinical pharmacist) in order of total risk score. Individual item scores for each triggered flag can also be displayed. By selecting a patient’s name, the user can access a new page with a graphical representation of the total ART score at the time of each assessment (Figure 2).
Reports showing the numbers of admissions in each risk group for a specific period of time can be generated for monitoring of key performance indicators such as the proportion of cases in which medication reconciliation is conducted. The reporting functionality could be filtered by ward or admitting specialty service.
Clinical pharmacists were provided with training prior to the ART go-live date in October 2011. This was initially done as an education session to the department and later incorporated into the clinical pharmacist training package for new staff. Staff training emphasized that the prioritization tool must always be used in conjunction with clinical judgment. After ART implementation, regular feedback was sought from clinical pharmacists regarding their experience with the tool and areas for improvement.
Monthly electronic reports showing the numbers of high-, medium-, and low-risk admissions are merged with medication reconciliation reports to provide departmental key performance indicators. This was found to be a particularly useful function; previously, there was no method to measure the impact of medication safety initiatives on different patient populations.
Impact of ART
With the introduction of the ART, the clinical pharmacy service adopted a decentralized practice model, with pharmacists assigned to specific medical teams. The combination of the ART and teambased pharmacist deployment has facilitated the provision of improved clinical services, such as electronic medication reconciliation and clinical review, to high-risk patients.
The ART also enabled a new pharmacy initiative at our hospital: targeting high-risk patients for discharge support services, which include medication reconciliation, counseling at discharge, and community follow-up. Over one eight-month period, 765 high-risk patients were prioritized for discharge services and 526 medication errors were prevented, 174 of which were serious enough (as determined via review by two senior clinical pharmacists in the department) to pose a threat of moderate-to-major patient harm requiring treatment or readmission.
Since ART implementation, the total number of high-risk patients receiving electronic medication reconciliation and review by the pharmacists almost doubled, from an average of 280 per month (October 2010 through September 2011) to an average of 500 per month (October 2011 through September 2012). Overall, at the time of writing, an average of 1800 patients at all risk levels (including medium- and low-risk patients) had received electronic medication reconciliation and review monthly, with approximately 30% identified as being at high risk for ADEs.
Clinical pharmacists’ feedback regarding the ART has been very positive, citing ease of use, time savings (an average of one hour daily relative to the time previously required for preparation of patient work lists), and an improved ability to provide early interventions to the patients in greatest need and better tailor interventions for other patients. Furthermore, as ART information is updated three times daily, newly admitted high-risk patients and patients whose clinical status has changed and now warrants a high-risk ART score are seen by a pharmacist promptly; previously, they might not have been identified or seen until the following day.
We have demonstrated that the use of an electronic prioritization tool is feasible and can improve clinical pharmacist workflow and, potentially, patient outcomes.
The ART includes a total of 38 flags, chosen from the international literature and adapted to suit local experience. Although we do not have complete electronic medical records, all the ART flags were derived from routinely used electronic databases; therefore, applying the ART to prioritize patients requires no extra work on the part of pharmacists. The user interface not only displays the patient’s risk score but also identifies specific components of risk and the change in risk over the course of the admission.
The ART identifies a cohort of potentially at-risk patients for analysis, validation, and evaluation. The ART provides a near real-time monitoring system that enables pharmacists to consistently review their patient lists, with patients queued in order of priority for intervention, as recommended by the Pharmacy Practice Model Initiative (PPMI) Summit report.31 According to the key enablers identified in the recommendations of the 2010 PPMI Summit, including those specifying the technology needed to optimally deploy pharmacist resources, the ART met most of the recommendations for an ideal patient prioritization system.32
After a review of the literature, we believe that the ART is the first electronic pharmacy patient prioritization tool of such complexity to be used in a large hospital. Most of the prioritization tools identified in our review used a manual process and, relative to the ART, a more limited set of risk assessment criteria18,19,33; others were more conceptual in nature.34
Limitations and future directions
Although face validity testing has shown the tool to correctly identify patients at risk for MEs and ADEs, formal validation to determine that the tool correctly prioritizes patients as being at high, medium, or low ADE risk is yet to be completed.
Partial validation of the tool’s sensitivity in identifying at-risk patients using an adapted version of the IHI’s ADE trigger tool has been considered. In such a validation study, the adapted trigger tool would be used to evaluate cohorts of high- and low-risk patients (as defined by the ART tool) and a third randomly selected control cohort; the resulting ART predictions would be compared with documented rates of actual ADEs and events with the potential to harm patients (i.e., identified MEs) in the three groups during hospital stays. This sort of validation would be challenging given the large number of patient admission reviews that would be required for statistical validation. Alternatively, the data generated through the use of the IHI ADE trigger tool process over a period of years could be evaluated to provide insights to guide the refinement of the risk flags that comprise the ART tool in order to better target patients most likely to experience harm within our institution.
It should also be noted that the increase in clinical pharmacist efficiency observed since the ART was implemented might be at least partly due to other changes in pharmacy practice during the implementation time frame, such as a move to a decentralized practice model and a change in the pharmacist staffing level (an increase of 4 FTEs over the two years of ART development).
Since implementation, the tool has received a great deal of national interest from other hospitals in New Zealand, as evidenced by the voluntary attendance of 25 senior hospital pharmacists from across the country at an ART flags validation meeting. During this meeting, it became evident that many hospital pharmacy departments struggle with manual patient prioritization and would benefit from an electronic tool, such as the ART, to help improve workflow efficiency for clinical pharmacists and aid medication safety efforts.
At the time of writing, formal validation of the tool using two methodologies, as well as the collection of outcomes data, was ongoing.
By facilitating the identification and monitoring of patients at high risk for MEs and ADEs, the ART has enabled one hospital’s clinical pharmacists to conduct interventions such as medication reconciliation and clinical review in a more timely and targeted manner.
The authors acknowledge the clinical pharmacists and the team at the Centre for Quality Improvement, Middlemore Hospital, for their contributions.
The authors have declared no potential conflicts of interest.
- Copyright © 2014 by the American Society of Health-System Pharmacists, Inc. All rights reserved.