Skip to main content
Download PDF
Download ePub
  • Systematic review
  • Open access
  • Published:

Identifying barriers and facilitators to successful implementation of computerized clinical decision support systems in hospitals: a NASSS framework-informed scoping review

Implementation Science volume 18, Article number: 32 (2023) Cite this article

Abstract

Background

Successful implementation and utilization of Computerized Clinical Decision Support Systems (CDSS) in hospitals is complex and challenging. Implementation science, and in particular the Nonadoption, Abandonment, Scale-up, Spread and Sustainability (NASSS) framework, may offer a systematic approach for identifying and addressing these challenges. This review aimed to identify, categorize, and describe barriers and facilitators to CDSS implementation in hospital settings and map them to the NASSS framework. Exploring the applicability of the NASSS framework to CDSS implementation was a secondary aim.

Methods

Electronic database searches were conducted (21 July 2020; updated 5 April 2022) in Ovid MEDLINE, Embase, Scopus, PyscInfo, and CINAHL. Original research studies reporting on measured or perceived barriers and/or facilitators to implementation and adoption of CDSS in hospital settings, or attitudes of healthcare professionals towards CDSS were included. Articles with a primary focus on CDSS development were excluded. No language or date restrictions were applied. We used qualitative content analysis to identify determinants and organize them into higher-order themes, which were then reflexively mapped to the NASSS framework.

Results

Forty-four publications were included. These comprised a range of study designs, geographic locations, participants, technology types, CDSS functions, and clinical contexts of implementation. A total of 227 individual barriers and 130 individual facilitators were identified across the included studies. The most commonly reported influences on implementation were fit of CDSS with workflows (19 studies), the usefulness of the CDSS output in practice (17 studies), CDSS technical dependencies and design (16 studies), trust of users in the CDSS input data and evidence base (15 studies), and the contextual fit of the CDSS with the user’s role or clinical setting (14 studies). Most determinants could be appropriately categorized into domains of the NASSS framework with barriers and facilitators in the “Technology,” “Organization,” and “Adopters” domains most frequently reported. No determinants were assigned to the “Embedding and Adaptation Over Time” domain.

Conclusions

This review identified the most common determinants which could be targeted for modification to either remove barriers or facilitate the adoption and use of CDSS within hospitals. Greater adoption of implementation theory should be encouraged to support CDSS implementation.

Peer Review reports

Background

Clinical Decision Support Systems (CDSS) are digital tools that can assist healthcare providers in making clinical decisions by providing targeted information [1]. They have the potential to improve various aspects of healthcare delivery, such as patient safety, clinical management, diagnostic support, cost management, and administrative efficiency [2]. CDSS can take different forms, such as guideline-based algorithms for chronic disease risk and screening, personalized medication dosing, and alerts for potential or historical adverse events [3, 4].

Despite the varied applications and benefits of CDSS, there are challenges related to their adoption, appropriate use, and sustainment over time in clinical settings [5, 6]. Challenges include poor uptake, inappropriate use, unintended consequences, and abandonment of the technology over time [5, 7,8,9]. The reasons for these challenges are multifactorial and include the social and organizational complexity of large health systems which makes it difficult to plan for downstream consequences with confidence [10, 11]. CDSS implementation is also affected by unforeseen system modifications to adapt to fast changing contexts [12]. Ideally, hospital-system and research stakeholders need to identify constructs that have been shown to influence implementation processes and outcomes, understand potential mechanisms of change, and identify potential barriers and facilitators to successful implementation [13, 14]. This is important as contextually informed implementation strategies targeting clinician behavior are a stronger predictor of CDSS uptake than technological design and content features [6]. Consequently, evidence-based guidance to inform and plan for such contextually appropriate implementation is necessary for greater systematic success of technological interventions in healthcare systems [5,6,7,8,9,10,11,12].

Implementation science offers theory-informed systematic processes for research translation [15]. Its theories, models, and frameworks help to identify barriers and facilitators throughout the implementation process and at different contextual levels. This includes both the immediate context of individuals receiving health-related interventions, their healthcare providers, healthcare organizations, and other stakeholders involved in care delivery, as well as the broader political, economic, and social setting [16]. Yet, few previous reviews of CDSS have applied a comprehensive implementation science framework within the context of the implementation processes [2, 3, 6].

The Nonadoption, Abandonment, Scale-up, Spread and Sustainability (NASSS) framework was developed to understand the successful implementation of technology-supported health (and social) care programs [17]. It positions technological interventions as part of a complex system [18] and has been used to guide implementation efforts and identify factors that influence implementation success in digital health [19,20,21]. A previous systematic review used the NASSS framework to assess contextual factors that can influence implementation success of virtual care in primary care settings [22]. The authors concluded that applying the NASSS framework helped to identify key contextual factors that can influence implementation success.

There is a substantial amount of literature which appears to examine different aspects of the implementation process for CDSSs within hospitals [3, 23, 24]. However, there have been limited attempts to classify these findings according to a deterministic implementation framework like NASSS. Such an approach may identify generalizable determinants to scale CDSS implementation efforts across varying hospital settings, integrate implementation processes into a unified framework, and inform more synergetic and comparative CDSS implementation research moving forward.

This study aimed to apply a qualitative reflexive analysis to identify, classify, and describe the potential barriers and facilitators, identified within peer-reviewed literature, that influence the implementation process (adoption, uptake, and use over time) of CDSS in hospital settings and map these findings to the NASSS framework. A secondary aim was to explore the applicability of the NASSS framework to CDSS implementation within a hospital context.

Method

A scoping review methodology was deemed appropriate to address the aims of this study as our goal was not to evaluate the evidence, but to identify and map key determinants of CDSS implementation and identify knowledge gaps [25]. This scoping review was conducted in accordance with the recommendations of Arksey and O’Malley [26] and Levac, Colquhoun [27]. Step (1) involved identifying the research question as “What are the barriers and facilitators to implementation of CDSS in hospital settings,” with the sub-question of “How can the determinants of CDSS implementation in hospital settings be categorized using an implementation science framework.” The remaining review steps consisted of (2) identifying relevant studies, (3) selecting studies, (4) charting the data, and (5) collating, summarizing, and reporting the results. The reporting of this scoping review was guided by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) checklist [28]. A protocol for this review and mapping was developed by BA before commencing searches but was not published.

Identifying relevant studies

We searched electronic databases on 21 July 2020 and again on 5 April 2022. The databases searched were Ovid MEDLINE (via EBSCOhost), Embase, Scopus, PyscInfo, and CINAHL (via EBSCOhost). The search strategy was iteratively created and refined using the SearchRefinery tool [29] with MeSH terms and keywords related to CDSS, implementation, and hospital settings (see Additional file 1). The final search string is provided in Table 1. The Polyglot Search Translator [30] was used to translate the Ovid MEDLINE string across all other databases, which were also searched for additional subject headings. The final search results were exported into EndNote [31], duplicates removed, and remaining records uploaded into Rayyan [32] for screening. Generation of the search terms, execution of both searches, and deduplication of records were all conducted by the same author (BA) in consultation with a medical librarian.

Table 1 Final search string used to identify articles for the study
Full size table

Study selection

Inclusion and exclusion criteria were determined prior to the database searches. For inclusion, articles were required to be (1) published, peer-reviewed original research studies of qualitative, quantitative or mixed methods design, (2) prospectively or retrospectively reported on measured or perceived barriers and/or facilitators to implementation and adoption of CDSS, (3) based in hospital settings. Studies of similar and potentially overlapping concepts, including (4) attitudes of healthcare professionals towards CDSS, were also included. All health conditions, hospital settings (inpatient and outpatient), and patient groups were eligible.

For this review CDSS was defined as any technology system designed to provide health professionals or operational staff with information/decision support that was filtered or targeted to a specific user, patient, or situation [33]. All decision support typologies, including both knowledge-based and non-knowledge-based systems, were included. There were no exclusion criteria related to the ways by which the users engaged with the decision support information; for example, dashboards, interruptive alerts, or reminder systems were all considered within scope.

Articles were excluded if they focused on consumer/patient facing decision support, e.g., decision aids, or primarily on CDSS development and not implementation. No language or date restrictions were applied, but papers unobtainable in full-text format (e.g., conference abstracts) were excluded.

Preliminary screening was piloted by all reviewers with a small sample of records to improve consistency of study selection criteria. The titles and abstracts of all records were then independently co-screened in Rayyan by eight reviewers working in pairs. Coding pairs met with a third reviewer (BA) to reach consensus about disagreements. Records that proceeded to full-text review were obtained and independently co-screened by two reviewer pairs (BA and SN, TD and DR). Disagreements about the inclusion of articles between reviewers in each pair were discussed in meetings facilitated by a third reviewer from the other pair until consensus was reached. As recommended by Levac, Colquhoun, and O’Brien [27], the core multidisciplinary review team (BA, SN, TD, and DR) met regularly to discuss any uncertainties related to study selection or review scope.

Charting the data

A shared online data extraction form was created using Microsoft Excel and was piloted and refined with several included studies. Subsequently, the following information was extracted from each included article independently by two authors (TD, DR): study identifiers (e.g., year, author, country), study type (e.g., observational, RCT), research methodology (e.g., quantitative, qualitative, mixed methods), participant descriptors (e.g., sample size, professional discipline), description of CDSS, clinical or organizational setting of CDSS (e.g., ward, hospital or multiple sites), key barriers to implementation/adoption, key facilitators to implementation/adoption, and any other details noted as potentially important for the study (e.g. implementation processes or frameworks). To address discrepancies in the extracted data, reporting issues, and data verification, three authors (TD, DR, SN) held meetings and iteratively updated the extraction form. In these cases, the relevant manuscript was thoroughly re-examined, specifically referring to each variable in the extraction document, until a consensus was reached.

Collating, reporting, and summarizing the results

There were several steps in collating and reporting the results which were conducted iteratively in a series of day-long workshops attended by four of the authors (TD, DR, BA, SN).

Data analysis

Our approach involved basic qualitative content analysis of all included publications (qualitative, quantitative, and mixed methods) using open coding and allocation into higher-order categories [34]. First, using the data extraction Excel sheet, each barrier and facilitator was assigned a short summarizing phrase or word that meaningfully captured its essence. These initial codes were discussed and revised with some original codes being discarded, amended, or subsumed by other codes to create higher-order codes. Following this, the content of each higher-order code and the underlying meaning of each was examined and discussed. The outcome of this process was to reflexively map each of the higher-order codes onto one or more of the NASSS domains and sub-domains with which they aligned.

The analysis was primarily deductive with the intent was to align barriers and facilitators identified in the literature with pre-existing domains within the NASSS framework. It is important to note that, in the analysis, no restriction was placed on the number of domains or sub-domains with which an individual barrier or facilitator could align. This was because the object of the study was not to strictly categorize barriers and facilitators using the NASSS but to understand the role of such a theoretical framework in capturing the complexity of CDSS implementation and adoption within real-world hospital systems.

A descriptive numerical summary of all included studies, and the individual barriers and facilitators to CDSS implementation was collated. We also developed a visual matrix including each study’s barriers and facilitators mapped across NASSS domains to demonstrate common findings and identify gaps in the research. Finally, a tree map was created to visualize the most frequently reported determinants to CDSS implementation and adoption in the included studies.

Results

After deduplication, 2163 articles were identified in the database searches. Based on title and abstract screening, 1983 of these were excluded. The remaining 181 full-text publications were retrieved and assessed for eligibility, resulting in the exclusion of 137 studies. The 44 remaining publications were included in this scoping review. The article selection process and reasons for exclusion are presented in Fig. 1.

Fig. 1
figure 1

Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow diagram for article selection

Full size image

Study characteristics

The characteristics of the 44 included studies, including their aim, participants, and CDSS type, are presented in Table 2. The studies were published between 2002 and 2022 with approximately 70% published since 2015. Most studies were published by authors residing in North America (n = 24). There were a mix of study designs, although most employed qualitative or mixed methods (n = 38). Ten studies were conducted as a precursor to CDSS implementation with the remainder completed during (peri) or post-implementation. CDSS were implemented across a wide range of clinical contexts and conditions (e.g., pulmonary embolism), uses (e.g., antimicrobial stewardship), patient cohorts (e.g., older persons), or combinations of these (e.g., imaging for pediatric traumatic brain injury). However, over one quarter of studies (n = 12) focused on CDSS intended for use in hospital emergency departments. Participants in the studies were overwhelmingly those engaged in clinical or patient-facing roles (e.g., physicians, nurses, pharmacists) with almost all studies (n = 40) clearly identifying individuals occupying these roles as participants.

Table 2 Characteristics of studies included in the scoping review and evidence map
Full size table

Summary of facilitators and barriers to CDSS implementation (determinants)

A visual summary of the analysis, synthesis, and reporting process we undertook to move from individual barriers and enablers [A] to higher-order codes [B] and unique determinants [C] is presented in Additional file 2. In total, 227 individual barriers and 130 individual facilitators were reported across the 44 included studies (implementation determinants, n = 357). Sixty-five higher-order codes were deductively constructed from these barriers and facilitators during qualitative content analysis and data synthesis. Fifty of these codes (77%) could be represented as 25 matched pairs which acted as either a barrier or facilitator depending on the study and implementation context. For example, CDSS implementation was facilitated when stakeholders perceived benefits to its use but hindered when they perceived it brought little relative advantage to usual practice (determinant: perceived benefits of CDSS). Other codes were only reported to act as barriers (n = 12, e.g., CDSS not sensitive to complexity) or only as facilitators (n = 3, e.g., CDSS used in audit, feedback, and benchmarking).

Combined, these higher-order codes represented 40 unique determinants influencing the implementation and use of CDSS in hospital settings (Fig. 2). The most commonly reported determinants were the fit of CDSS with workflows (n = 19, 44% of studies), the usefulness of the CDSS output in practice (n = 17, 40% of studies), CDSS technical dependencies and design (n = 16, 37% of studies), trust of users in the CDSS input data and evidence base (n = 15, 35% of studies), and the contextual fit of the CDSS with the user’s role or clinical setting (n = 14, 33% of studies).

Fig. 2
figure 2

Number of included studies reporting each higher-order determinant for CDSS implementation and adoption

Full size image

Fourteen of these higher-level determinants were reported in 25% or more of the included studies (Fig. 3). While these common determinants occurred across almost all NASSS domains, the CDSS “technology” and “value proposition” domain contained the largest concentration of reported barriers and facilitators.

Fig. 3
figure 3

Tree map highlighting most common CDSS implementation determinants (reported in 25% or more of studies) mapped to NASSS framework domain. Numbers represent the number of studies reporting the determinant

Full size image

Mapping barriers and facilitators to the NASSS framework

Mapping the barriers and facilitators identified in each individual study (Additional file 2A) to one of the seven domains of the NASSS framework enabled an examination of the key areas, and potentially understudied or underreported gaps, for CDSS implementation. This is visualized as a matrix in Fig. 4. Overall, barriers and facilitators reported in individual studies were most often aligned with the NASSS domains of “Technology” (n = 140, 35%), “Organization” (n = 108, 27%), and “Adopters” (n = 73, 18%). No codes were assigned to the “Embedding and Adaptation Over Time” domain. More barriers than facilitators were identified related to the “Condition/Context,” “Technology,” and “Adopters.” However, for the “Value Proposition,” “Organization,” and “Wider System,” just as many facilitators as barriers were noted. A detailed summary of the barriers and facilitators related to each NASSS domain and represented in Fig. 4 is discussed below.

Fig. 4
figure 4

Matrix demonstrating the frequency and distribution of barriers and facilitators reported in each study across NASSS domains and sub-domains

Full size image

Appropriateness of NASSS framework for categorizing CDSS implementation determinants

We experienced challenges mapping some codes to suitable NASSS sub-domains. In several instances, a code was applicable to a NASSS domain but the available sub-domains were not adequate to capture the description of the barrier or facilitator. For example, in one study [65], CDSS use was hindered by having limited available service options in the community for onward patient care after a clinical decision was recommended. While this could be coded as a challenge in the “Wider System” domain, we felt it only weakly aligned with the “interorganizational networking” sub-domain we mapped it to. This was also the case for some codes aligning with the “Adopters” category, where the nuance of psychological and motivational factors related to end users was not adequately captured in the “Staff” sub-domain. Additionally, we chose to interpret the “Condition” sub-domain of “Nature of Condition” as “Clinical Context” where barriers or facilitators may have been specific to the complexity of a particular clinical area (e.g., the emergency department).

Condition domain

This domain refers to the contextual factors and characteristics of the healthcare condition or problem for which a particular intervention or technology is being implemented. Twenty-five studies (57%) reported at least one barrier or facilitator which could be categorized in the “Condition” domain. The condition or clinical context was most often reported to act as a barrier to CDSS implementation. The most common barriers included the inflexibility of CDSS to allow for tailoring to contextual and situational factors specific to a particular illness, clinical context, or patient cohort and the difficulties associated with “defining complex clinical situations within a set of rules” [50]. These were a particular concern for CDSS deployed in complex, fast-paced emergency departments [36, 65, 66, 68] for specialist clinicians, and in intensive care units [67]. Where CDSS information and recommendations could be tailored to local practice, however, implementation was facilitated.

Technology domain

The technology domain captured the material features, knowledge inputs and outputs of the CDSS, its procurement model, and the knowledge and skill required to use it. Almost all studies reported barriers or facilitators related to the CDSS technology itself (n = 40, 91%) and it consistently acted as a barrier across all sub-domains.

Many material and technical features inherent within implemented CDSS led to limitations and challenges in how users adopted the tools in practice. These included poor user experience, issues with the interface or physical design, alert fatigue, information redundancy, privacy concerns, technical issues, and inconsistent performance. Other issues related to the lack of CDSS integration within existing clinical pathways and limited interoperability within existing technological architecture, both requiring workarounds for successful implementation. By contrast, ensuring CDSS integrated easily into existing systems and employing user-informed design principles in its creation were both highlighted as key facilitators to implementation.

An important aspect to consider during the implementation of CDSS is the type and quality of knowledge or information that the technology provides to end users. Studies indicated clinicians held concerns about the completeness, quality, accuracy, and relevance of the information used to inform the decision support (e.g., evidence or guideline in knowledge-based CDSS). This often led to a mistrust of the CDSS output and lack of acceptance of the decision made. Related to this were wider issues of CDSS design such as possibilities for bias inherent in the data used (e.g., population-specific) or the level of validation required to ensure Artificial Intelligence and algorithms are sufficiently equitable. For example, consider an algorithm used to detect lung cancer from medical imaging scans. If this algorithm is trained on a dataset consisting mainly of scans from male patients, it may not perform as well when applied to scans from female patients. This gender bias in the training data can lead to disparities in the algorithm’s accuracy, potentially resulting in missed or delayed lung cancer diagnoses in women. Additionally, it is crucial to determine the appropriate level of validation needed to ensure that the integration of Artificial Intelligence and algorithms in CDSS is fair and unbiased. This validation process should address potential algorithmic biases, such as disparities in diagnostic accuracy or treatment recommendations based on race or gender. Additionally, in several studies, the CDSS output was perceived to have limited usefulness in practically delivering care. For example, one output was described as “not prescriptive enough to effectively apply to individual patients at the point-of-care” [62]. However, this technology sub-domain could equally act as an important implementation facilitator. Studies observing this sub-domain as a facilitator reported clear evidence-based inputs underpinning the CDSS to build clinician trust; a CDSS output which could be used to effectively supplement clinical expertise or judgment; and/or an output which could be used as a tool to foster communication with patients, and between teams of clinicians.

Fewer studies reported determinants related to skills or knowledge of end users, and most of these were barriers. Challenges were related to a lack of user knowledge of the CDSS system or clinical condition supported, coupled with limited or poor technical training and support. Perceptions that using a new CDSS would result in a loss of productivity could also deter end users from engaging with the CDSS. Two studies [62, 71] highlighted such impacts on productivity for clinicians due to the learning curve of a new technology, “when clinical pathways were used, providers initially struggled with the order sets. Surgeons had increased order entry time and a decrease in productivity” [62].

Value proposition domain

This domain explores the proposition or value that the intervention offers to users, providers, organizations, and the wider healthcare system. It is important to consider how, and for whom, a new CDSS technology generates value to ensure its successful implementation. Almost two-thirds of studies in this review (n = 28) reported the perceived relative advantage of using CDSS compared to usual practice was an equally important barrier and facilitator of uptake. That is, CDSS use was facilitated if it was perceived to enhance the ability of clinicians to work effectively or efficiently, improve workflow and save time, or contribute to positive patient outcomes. Additionally, if clinicians saw benefit in the role of CDSS to standardize implementation of guidelines or care, it was more readily accepted. Conversely, if the CDSS was not useful or did not deliver benefit of some kind, whether real or perceived, then it would be unlikely to be accepted. Several studies [12, 46, 50] also cited the risk of medical errors and patient safety concerns as diminishing the value of the technology.

Adopter system domain

The adopter system refers to the individuals, groups, or organizations that are involved in the adoption, implementation, and use of a particular innovation or intervention within the healthcare setting. Thirty-six studies (82%) reported key considerations of CDSS adoption related to its users. Determinants in this domain more often acted as barriers than facilitators; however, some key enabling factors were noted. While the NASSS framework considers staff, patients, and caregivers in this domain, the studies in this review almost exclusively focused on healthcare professionals. The key themes related to perceptions of impact of CDSS on their role, responsibility, and professional autonomy; personal factors such as general attitudes and motivations; and their physical capability and opportunity to use the CDSS. Only two papers [37, 70] highlighted barriers which were perceived to impact patient adoption of CDSS. The main concerns were the requirement of patients to enter data into the system, and patients who did not act on recommendations provided by the clinician and CDSS.

Staff perceptions of professional autonomy were perceived to be either reduced or enhanced by CDSS. This depended on whether CDSS was seen as a tool that inhibited or facilitated clinician ability to make judgments. While implementation and adoption were hindered when clinicians had “fear of losing control over autonomous decision-making” [12], where local implementation allowed professional autonomy to be balanced with the standardization provided by the CDSS output, adoption was facilitated. Similarly, perceptions of professional role identity and responsibility were both positive and negative influences on implementation. CDSS were seen as enabling professionals of different specialties to engage in the decision-making domains of others. This was seen as a barrier in some specialties where there may be clearly defined hierarchies, but as a facilitator where CDSS could reinforce the tasks and responsibilities of one particular profession (e.g., Doctor) compared to another (e.g., Pharmacist).

Perceptions, irrespective of their accuracy, about the time and effort required to use a CDSS, its complexity, availability, usability, timeliness, or usefulness in particular work contexts were reported to reduce user motivation for CDSS. Lack of acceptance, intentional passive resistance, and cognitive influences (e.g., memory and attention) often meant users reverted to workflows and processes they were more comfortable with. Psychological characteristics of individual end users also emerged as barriers to the adoption and implementation of CDSS. This included individual preferences and attitudes about the features, usability, or perceived benefits of a CDSS. The level of risk tolerance exhibited by individual clinicians can also influenced their willingness to trust and therefore use CDSS. Perceptions about CDSS being less accurate also led to user hesitancy among clinicians. Additionally, the perceived relationship with the patient also adversely impacted uptake of CDSS. Healthcare professionals who highly value personal interactions and direct patient engagement harbored concerns about the potential disruption of the doctor-patient relationship that may arise with the use of technology-driven interventions.

Finally, although reported to a lesser extent, capability barriers incorporated issues such as lack of knowledge, skills, or familiarity with the CDSS. Physical opportunity barriers related to limited time to invest in learning and using the new CDSS.

Organization domain

This domain considered organizational capacity and readiness to implement CDSS, funding and costs of the new technology, implementation processes, and changes to team interactions and routines. Interestingly, the 35 studies (80%) categorized in this domain reported barriers and facilitators to occur almost equally.

One of the most frequently reported considerations in the organizational domain was the extent of change needed to organizational routines to implement a new CDSS. More likely to act as a barrier in included studies, this was reflective of CDSS fit with existing workflows, and existing governance practices. CDSS implementation that required minimal workflow changes, interruptions, or unnecessary duplication of activities were more likely to be successful.

An early, ongoing, and supported implementation plan was also reported as one of the most frequent facilitators of CDSS implementation in the organization domain. Components of successful implementation strategies included local champions and super-users that promoted and supported use of the CDSS; benchmarking, audit, and feedback to drive change; provision of technical training and support; and early engagement and involvement of users in the development and implementation of the CDSS. Conversely, a lack of planned implementation process was reported by other studies to lead to poor organizational preparedness and limited buy-in which hindered implementation.

Organizations that demonstrated a good overall capacity to innovate provided enabling environments for CDSS implementation. This incorporated a positive institutional culture, clear innovation strategy, organizational support for change, and previous successful technological deployment. Conversely, limited leadership buy-in for innovation or a lack of organizational policies or goals to innovate created barriers for CDSS implementation. Some organizations were reported to have an ingrained culture that was generally resistant to change. Thus, the successful introduction of an innovation such as CDSS was unlikely to be successful unless this cultural factor was addressed. The readiness of the organization for technology-supported change was more often reported to be a barrier in included studies, highlighting such concerns as limitations in the system-level infrastructure or information technology resources to enable CDSS implementation. Similarly, issues related to governance represented organizational barriers with respect to the readiness of policies, procedures, and reporting requirements for CDSS. Only one paper [59] specifically reported that consideration of increased costs of resources for the organization under conditions of uncertain funding could be a barrier to CDSS use.

Wider system domain

The wider system domain considers factors such as the political, economic, and social contexts within which the healthcare system operates. Only eight papers (18%) reported determinants of CDSS implementation which sat in the wider healthcare, social, or policy system. The majority of these (n = 6) were concerned with regulatory or medico-legal issues related to aspects of CDSS clinical use or compliance. For example, in two studies [12, 69], clinicians were concerned about where legal responsibility would lie if patient harm occurred when following CDSS recommendations. However, one study [68] reported CDSS would have better adoption if it provided data to help meet external regulatory requirements. Two studies [41, 69] reported on the enabling nature of professional bodies in CDSS implementation. This was because they could provide guidelines to support CDSS input, provide a platform for knowledge exchange, or lead an implementation agenda to increase use in different contexts and professions. Finally, pressure on healthcare budgets was identified as a facilitator for the adoption of CDSS in one paper [69] when there was an expectation that use of such technologies would lead to efficiencies or increased effectiveness and therefore reductions in costs.

Discussion

Our study identified a comprehensive set of determinants prioritized by the literature that impact the implementation process of CDSS within hospital systems. These were captured across a mix of study designs, geographic locations, participants, technology types, CDSS functions, and clinical contexts of implementation. In doing so we have systematically identified common barriers and facilitators which can inform the design of targeted implementation strategies to enhance the adoption of CDSS within hospitals. In particular, the findings suggest that there is a need for CDSS to be designed to fit the workflows and contexts of clinical practice and that enhancing user trust in the accuracy, value, and relevance of the CDSS and its output is crucial for effective implementation outcomes. This finding is supported by previous literature which has emphasized the importance of implementation context and attaining good “clinician-patient-system integration” when developing CDSSs [76]. This review highlights the importance of further integrating implementation science principles into CDSS intervention design [6].

Adopting a theory-informed implementation science approach to this review has also enabled an examination of the complexity of CDSS implementation in real-world healthcare settings. To our knowledge, this is the first review to systematically identify, classify, and map peer-reviewed literature about barriers and facilitators of hospital-based CDSS implementation using an implementation science framework. Two recent systematic reviews assessing the determinants of CDSS uptake have been limited to relatively linear descriptions or analysis of the issue (i.e., including a recent pooled analysis with extreme heterogeneity [6, 77]). This underscores the value of a framework-informed classification of these determinants which identifies multiple implementation influencers along the adoption-user-sustainment pathway to inform a cascade of targeted implementation strategies.

By addressing the common barriers and facilitators identified, stakeholders can employ context-relevant implementation strategies to more effectively promote and sustain CDSS implementation outcomes [78]. To achieve this, the authors recommend decision makers should involve end users in the design and testing phases of CDSS to ensure that the system meets contextual needs and is aligned with existing workflows. This may allow for more effective and accurate resource prioritization [13]. Transparency and systematic governance of CDSS may build trust in the data sources and the accuracy of outputs, ensuring the CDSS is being used appropriately and in the best interests of patients [8]. Providing ongoing support and training to end users can address perceptual and psychosocial determinants of CDSS acceptance, enabling users to effectively use the system [10, 11]. Tailoring implementation strategies to different service contexts may address organizational-level determinants related to workflow integration and ensure the CDSS is compatible with the hospital’s existing processes and infrastructure [7]. Advocating for legislative clarity regulating the use of CDSS for clinician decision-making can potentially address wider system barriers, providing clarity, and may boost confidence in the use of CDSS tools.

Our review supports the value of the NASSS framework for assessing context and guiding CDSS implementation. It offers a nuanced view of determinants, processes, and outcomes by considering the dynamic interplay between the system, the context, and the stakeholders. In our review, most determinants, with very few exceptions, could be appropriately categorized by the contextually specific domains within the NASSS framework. This illustrates the external validity of this framework to identify and meaningfully categorize CDSS implementation determinants within hospital settings. Similar validity of the NASSS framework has been observed in reviews examining the implementation of artificial intelligence in health care settings [76] and virtual care implementation within primary care [22]. However, as was the case in our mapping, the authors of that paper also experienced challenges aligning some of the determinants with existing NASSS sub-domains.

Consequently, these findings suggest that NASSS may need to be further contextualized to account for individual technologies. Greater clarification of existing domains and sub-domains is required to understand how implementation of different technologies may be categorized within the existing framework. For example, understanding whether the adaptations to “The Condition” we made in this review are consistent with the underlying theory of NASSS. Additionally, new evidence-based sub-domains (for example in “The Adopters” or “Wider Condition”) may be required to supplement the current NASSS framework. With such refinements, NASSS would have better utility for context assessment and development of targeted implementation strategies to enhance the adoption, integration, and sustainability of CDSS in hospitals.

Additionally, a noteworthy limitation of the literature identified is the lack of identified determinants mapped to the “embedding over time” domain of NASSS. This reflects an ongoing need to systematically evaluate and document how adoption factors around digital health interventions are adapted and changed over time. This could be partially due to the limited funding opportunities available for longitudinal studies that extend beyond the initial evaluation phase. Understanding these determinants are especially important for acute health services like hospitals which face dynamical and changing contextual pressures related to budgets, socio-political events, and natural disasters. Furthermore, we note that most studies focusing on healthcare stakeholders predominantly included frontline clinicians and other health workers. We recommend future work also include other informants involved in policy, informatics, and health management which may serve to unpack wider-domain factors with greater specificity.

Limitations of this review are excluding studies which only described intervention development or conducted context assessments without any explicit reference to a CDSS, which might adjust for contextual determinants during the intervention development process (informed by context assessments and related co-design strategies). Furthermore, while patient-reported factors were not explicitly excluded from this review, very few (n = 2) included studies reported on this perspective. These excluded perspectives may serve to further contextualize and potentially add to the determinants identified. It is currently unclear how and if health consumer perspectives are generally sought during the intervention roll-out phases of CDSS within hospitals, and this constitutes a significant gap in our knowledge of these potentially important determinants. It must also be noted that the reported health system determinants of CDSS adoption do not represent an exhaustive list, since the absence of evidence does not indicate evidence of absence.

Conclusions

Ongoing efforts are needed to categorize common factors within hospital systems that influence the adoption, integration, and sustainability of digital health solutions. This will enable the optimal integration of digital innovations into hospital settings, ensuring intended improvements in outcomes can be achieved and maintained. Applying established implementation science approaches, such as using theoretical frameworks and models, can enhance the efficiency and effectiveness of this process in a systematic and contextually appropriate manner. The findings of our review contribute to assisting healthcare stakeholders in identifying key determinants that can be modified to facilitate the adoption of Clinical Decision Support Systems (CDSS) in various hospital settings.

Availability of data and materials

The authors declare that the data supporting findings of this review are available within the paper.

Abbreviations

CDSS:

Computerized clinical decision support system

NASSS:

Nonadoption, Abandonment, Scale-up, Spread, and Sustainability

References

  1. Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med. 2020;3(1):17.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Dissanayake PI, Colicchio TK, Cimino JJ. Using clinical reasoning ontologies to make smarter clinical decision support systems: a systematic review and data synthesis. J Am Med Inform Assoc. 2020;27(1):159–74.

    Article  PubMed  Google Scholar 

  3. Hak F, Guimarães T, Santos M. Towards effective clinical decision support systems: a systematic review. PLoS One. 2022;17(8):e0272846.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  4. World Health Organization (WHO). WHO guideline: recommendations on digital interventions for health system strengthening. Geneva: WHO; 2019.

    Google Scholar 

  5. Laka M, Milazzo A, Merlin T. Why provision of clinical decision support (CDS) is not enough? Factors influencing the CDS adoption. Eur J Public Health. 2020;30(Supplement_5):ckaa165.223.

    Article  Google Scholar 

  6. Kouri A, Yamada J, Lam Shin Cheung J, Van de Velde S, Gupta S. Do providers use computerized clinical decision support systems? A systematic review and meta-regression of clinical decision support uptake. Implement Sci. 2022;17(1):21.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Laka M, Milazzo A, Merlin T. Factors that impact the adoption of Clinical Decision Support Systems (CDSS) for antibiotic management. Int J Environ Res Public Health. 2021;18(4):1901.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Petkus H, Hoogewerf J, Wyatt JC. What do senior physicians think about AI and clinical decision support systems: quantitative and qualitative analysis of data from specialty societies. Clin Med (Lond). 2020;20(3):324–8.

    Article  PubMed  Google Scholar 

  9. Stone EG. Unintended adverse consequences of a clinical decision support system: two cases. J Am Med Inform Assoc. 2018;25(5):564–7.

    Article  PubMed  Google Scholar 

  10. Cresswell KM, Lee L, Mozaffar H, Williams R, Sheikh A, Robertson A, et al. Sustained user engagement in health information technology: the long road from implementation to system optimization of computerized physician order entry and clinical decision support systems for prescribing in hospitals in England. Health Serv Res. 2017;52(5):1928–57.

    Article  PubMed  Google Scholar 

  11. Perski O, Short CE. Acceptability of digital health interventions: embracing the complexity. Transl Behav Med. 2021;11(7):1473–80.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Liberati EG, Ruggiero F, Galuppo L, Gorli M, González-Lorenzo M, Maraldi M, et al. What hinders the uptake of computerized decision support systems in hospitals? A qualitative study and framework for implementation. Implement Sci. 2017;12(1):113.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Donovan T, Abell B, Fernando M, McPhail SM, Carter HE. Implementation costs of hospital-based computerised decision support systems: a systematic review. Implement Sci. 2023;18(1):7.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Iyamu I, Gómez-Ramírez O, Xu AX, Chang H-J, Watt S, Mckee G, et al. Challenges in the development of digital public health interventions and mapped solutions: findings from a scoping review. Digit Health. 2022;8:20552076221102256.

    PubMed  PubMed Central  Google Scholar 

  15. Bauer MS, Kirchner J. Implementation science: what is it and why should I care? Psychiatry Res. 2020;283:112376.

    Article  PubMed  Google Scholar 

  16. Peters DH, Adam T, Alonge O, Agyepong IA, Tran N. Republished research: implementation research: what it is and how to do it. Br J Sports Med. 2014;48(8):731.

    Article  PubMed  Google Scholar 

  17. Greenhalgh T, Wherton J, Papoutsi C, Lynch J, Hughes G, A’Court C, et al. Beyond adoption: a new framework for theorizing and evaluating nonadoption, abandonment, and challenges to the scale-up, spread, and sustainability of health and care technologies. J Med Internet Res. 2017;19(11):e367.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Greenhalgh T, Wherton J, Papoutsi C, Lynch J, Hughes G, A’Court C, et al. Analysing the role of complexity in explaining the fortunes of technology programmes: empirical application of the NASSS framework. BMC Med. 2018;16(1):66.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Abimbola S, Patel B, Peiris D, Patel A, Harris M, Usherwood T, et al. The NASSS framework for ex post theorisation of technology-supported change in healthcare: worked example of the TORPEDO programme. BMC Med. 2019;17(1):233.

    Article  PubMed  PubMed Central  Google Scholar 

  20. KadesjöBanck J, Bernhardsson S. Experiences from implementation of internet-delivered cognitive behaviour therapy for insomnia in psychiatric health care: a qualitative study applying the NASSS framework. BMC Health Serv Res. 2020;20(1):729.

    Article  Google Scholar 

  21. Grady A, Barnes C, Wolfenden L, Lecathelinais C, Yoong SL. Barriers and enablers to adoption of digital health interventions to support the implementation of dietary guidelines in early childhood education and care: cross-sectional study. J Med Internet Res. 2020;22(11):e22036.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Darley S, Coulson T, Peek N, Moschogianis S, van der Veer SN, Wong DC, et al. Understanding how the design and implementation of online consultations affect primary care quality: systematic review of evidence with recommendations for designers, providers, and researchers. J Med Internet Res. 2022;24(10):e37436.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Khong PCB, Holroyd E, Wang W. A critical review of the theoretical frameworks and the conceptual factors in the adoption of clinical decision support systems. Comput Inform Nurs. 2015;33(12):555–70.

    Article  PubMed  Google Scholar 

  24. Khanijahani A, Iezadi S, Dudley S, Goettler M, Kroetsch P, Wise J. Organizational, professional, and patient characteristics associated with artificial intelligence adoption in healthcare: a systematic review. Health Policy Technol. 2022;11(1):100602.

    Article  Google Scholar 

  25. Munn Z, Peters MDJ, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol. 2018;18(1):143.

    Article  PubMed  Google Scholar 

  26. Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.

    Article  Google Scholar 

  27. Levac D, Colquhoun H, O’Brien KK. Scoping studies: advancing the methodology. Implement Sci. 2010;5(1):69.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–73.

    Article  PubMed  Google Scholar 

  29. Scells H, Zuccon G. searchrefiner: a query visualisation and understanding tool for systematic reviews. In: Proceedings of the 27th ACM International Conference on Information and Knowledge Management. Torino: Association for Computing Machinery; 2018. p. 1939–42.

  30. Clark JM, Sanders S, Carter M, Honeyman D, Cleo G, Auld Y, et al. Improving the translation of search strategies using the Polyglot Search Translator: a randomized controlled trial. J Med Libr Assoc. 2020;108(2):195–207.

    Article  PubMed  PubMed Central  Google Scholar 

  31. The Endnote Team. Endnote. Endnote X20 ed. Philadelphia: Clarivate; 2013.

    Google Scholar 

  32. Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan—a web and mobile app for systematic reviews. Syst Rev. 2016;5(1):210.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Sim I, Gorman P, Greenes RA, Haynes RB, Kaplan B, Lehmann H, et al. Clinical decision support systems for the practice of evidence-based medicine. J Am Med Inform Assoc. 2001;8(6):527–34.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  34. Pollock D, Peters MDJ, Khalil H, McInerney P, Alexander L, Tricco AC, et al. Recommendations for the extraction, analysis, and presentation of results in scoping reviews. JBI Evid Synth. 2022;21(3):520–32.

    Article  Google Scholar 

  35. Blanco N, O’Hara LM, Robinson GL, Brown J, Heil E, Brown CH, et al. Health care worker perceptions toward computerized clinical decision support tools for Clostridium difficile infection reduction: a qualitative study at 2 hospitals. Am J Infect Control. 2018;46(10):1160–6.

    Article  PubMed  Google Scholar 

  36. Flynn D, Nesbitt DJ, Ford GA, McMeekin P, Rodgers H, Price C, et al. Development of a computerised decision aid for thrombolysis in acute stroke care. BMC Med Inform Decis Mak. 2015;15(1):1–15.

    Article  Google Scholar 

  37. Hasnie AA, Kumbamu A, Safarova MS, Caraballo PJ, Kullo IJ. A clinical decision support tool for familial hypercholesterolemia based on physician input. Mayo Clin Proc Innov Qual Outcomes. 2018;2(2):103–12.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Melnick ER, Holland WC, Ahmed OM, Ma AK, Michael SS, Goldberg HS, et al. An integrated web application for decision support and automation of EHR workflow: a case study of current challenges to standards-based messaging and scalability from the EMBED trial. JAMIA Open. 2019;2(4):434–9.

    Article  PubMed  PubMed Central  Google Scholar 

  39. Mugabe KV. Barriers and facilitators to the adoption of artificial intelligence in radiation oncology: a New Zealand study. Tech Innov Patient Support Radiat Oncol. 2021;18:16–21.

    Article  Google Scholar 

  40. Ploegmakers KJ, Medlock S, Linn AJ, Lin Y, Seppälä LJ, Petrovic M, et al. Barriers and facilitators in using a Clinical Decision Support System for fall risk management for older people: a European survey. Eur Geriatr Med. 2022;13(2):395–405.

    Article  PubMed  Google Scholar 

  41. Westafer LM, Kunz A, Bugajska P, Hughes A, Mazor KM, Schoenfeld EM, et al. Provider perspectives on the use of evidence-based risk stratification tools in the evaluation of pulmonary embolism: a qualitative study. Acad Emerg Med. 2020;27(6):447–56.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Yadav K, Chamberlain JM, Lewis VR, Abts N, Chawla S, Hernandez A, et al. Designing real-time decision support for trauma resuscitations. Acad Emerg Med. 2015;22(9):1076–84.

    Article  PubMed  PubMed Central  Google Scholar 

  43. Ballard DW, Vemula R, Chettipally UK, Kene MV, Mark DG, Elms AK, et al. Optimizing clinical decision support in the electronic health record: clinical characteristics associated with the use of a decision tool for disposition of ED patients with pulmonary embolism. Appl Clin Inform. 2016;7(3):883–98.

    Article  PubMed  PubMed Central  Google Scholar 

  44. Bersani K, Fuller TE, Garabedian P, Espares J, Mlaver E, Businger A, et al. Use, perceived usability, and barriers to implementation of a patient safety dashboard integrated within a vendor EHR. Appl Clin Inform. 2020;11(1):034–45.

    Article  Google Scholar 

  45. Bowen SP, Johnson KMPA, Reed MHF, Zhang L, Curry LP. The effect of incorporating guidelines into a computerized order entry system for diagnostic imaging. J Am Coll Radiol. 2011;8(4):251–8.

    Article  PubMed  Google Scholar 

  46. Campion TR, Waitman LR, Lorenzi NM, May AK, Gadd CS. Barriers and facilitators to the use of computer-based intensive insulin therapy. Int J Med Inform (Shannon, Ireland). 2011;80(12):863–71.

    Article  Google Scholar 

  47. Chow A, Lye DCB, Arah OA. Psychosocial determinants of physicians’ acceptance of recommendations by antibiotic computerised decision support systems: a mixed methods study. Int J Antimicrob Agents. 2014;45(3):295–304.

    Article  PubMed  Google Scholar 

  48. Chua AQ, Tang SSL, Lee LW, Yii DYC, Kong ST, Lee W, et al. Psychosocial determinants of physician acceptance toward an antimicrobial stewardship program and its computerized decision support system in an acute care tertiary hospital. J Am Coll Clin Pharm. 2018;1(1):e1–8.

    Article  Google Scholar 

  49. Chung P, Scandlyn J, Dayan PS, Mistry RD. Working at the intersection of context, culture, and technology: provider perspectives on antimicrobial stewardship in the emergency department using electronic health record clinical decision support. Am J Infect Control. 2017;45(11):1198–202.

    Article  PubMed  Google Scholar 

  50. Collins I, Breathnach O, Felle P. Electronic clinical decision support systems attitudes and barriers to use in the oncology setting. Ir J Med Sci. 2012;181(4):521–5.

    Article  CAS  PubMed  Google Scholar 

  51. de Vries AE, van der Wal MHL, Nieuwenhuis MMW, de Jong RM, van Dijk RB, Jaarsma T, et al. Perceived barriers of heart failure nurses and cardiologists in using clinical decision support systems in the treatment of heart failure patients. BMC Med Inform Decis Mak. 2013;13(1):54.

    Article  PubMed  PubMed Central  Google Scholar 

  52. English D, Ankem K, English K. Acceptance of clinical decision support surveillance technology in the clinical pharmacy. Inform Health Soc Care. 2017;42(2):135–52.

    Article  PubMed  Google Scholar 

  53. Giuliano CA, Binienda J, Kale-Pradhan PB, Fakih MG. “I never would have caught that before”: pharmacist perceptions of using clinical decision support for antimicrobial stewardship in the United States. Qual Health Res. 2018;28(5):745–55.

    Article  PubMed  Google Scholar 

  54. Glassman PA, Simon B, Belperio P, Lanto A. Improving recognition of drug interactions: benefits and barriers to using automated drug alerts. Med Care. 2002;40(12):1161–71.

    Article  PubMed  Google Scholar 

  55. Goud R, van Engen-Verheul M, de Keizer NF, Bal R, Hasman A, Hellemans IM, et al. The effect of computerized decision support on barriers to guideline implementation: a qualitative study in outpatient cardiac rehabilitation. Int J Med Inform (Shannon, Ireland). 2010;79(6):430–7.

    Article  Google Scholar 

  56. Grau LE, Weiss J, O’Leary TK, Camenga D, Bernstein SL. Electronic decision support for treatment of hospitalized smokers: a qualitative analysis of physicians’ knowledge, attitudes, and practices. Drug Alcohol Depend. 2019;194:296–301.

    Article  PubMed  Google Scholar 

  57. Green TA, Whitt S, Belden JL, Erdelez S, Shyu C-R. Medical calculators: prevalence, and barriers to use. Comput Methods Programs Biomed. 2019;179:105002.

    Article  PubMed  Google Scholar 

  58. Gutenstein M, Pickering JW, Than M. Development of a digital clinical pathway for emergency medicine: lessons from usability testing and implementation failure. Health Informatics J. 2019;25(4):1563–71.

    Article  PubMed  Google Scholar 

  59. Jacobs J, Weir C, Evans RS, Staes C. Assessment of readiness for clinical decision support to aid laboratory monitoring of immunosuppressive care at U.S. liver transplant centers. Appl Clin Inform. 2014;5(4):988–1004.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  60. Johnson R, Evans M, Cramer H, Bennert K, Morris R, Eldridge S, et al. Feasibility and impact of a computerised clinical decision support system on investigation and initial management of new onset chest pain: a mixed methods study. BMC Med Inform Decis Mak. 2015;15(1):71.

    Article  PubMed  PubMed Central  Google Scholar 

  61. Lai F, Macmillan J, Daudelin DH, Kent DM. The potential of training to increase acceptance and use of computerized decision support systems for medical diagnosis. Hum Factors. 2006;48(1):95–108.

    Article  PubMed  Google Scholar 

  62. Lesselroth BJ, Yang J, McConnachie J, Brenk T, Winterbottom L. Addressing the sociotechnical drivers of quality improvement: a case study of post-operative DVT prophylaxis computerised decision support. BMJ Qual Saf. 2011;20(5):381–9.

    Article  PubMed  PubMed Central  Google Scholar 

  63. Masterson Creber RM, Dayan PS, Kuppermann N, Ballard DW, Tzimenatos L, Alessandrini E, et al. Applying the RE-AIM Framework for the evaluation of a clinical decision support tool for pediatric head trauma: a mixed-methods study. Appl Clin Inform. 2018;9(3):693–703.

    Article  PubMed  PubMed Central  Google Scholar 

  64. Miller MK, Mollen C, Behr K, Dowd MD, Miller E, Satterwhite CL, et al. Development of a novel computerized clinical decision support system to improve adolescent sexual health care provision. Acad Emerg Med. 2019;26(4):420–33.

    Article  PubMed  Google Scholar 

  65. Petitgand C, Motulsky A, Denis J-L, Régis C. Investigating the barriers to physician adoption of an artificial intelligence-based decision support system in emergency care: an interpretative qualitative study. In: Digital personalized health and medicine. Amsterdam: IOS Press; 2020. p. 1001–5.

    Google Scholar 

  66. Salwei ME, Carayon P, Hoonakker PL, Hundt AS, Wiegmann D, Pulia M, et al. Workflow integration analysis of a human factors-based clinical decision support in the emergency department. Appl Ergon. 2021;97:103498.

    Article  PubMed  PubMed Central  Google Scholar 

  67. Santucci W, Day RO, Baysari MT. Evaluation of hospital-wide computerised decision support in an intensive care unit: an observational study. Anaesth Intensive Care. 2016;44(4):507–12.

    Article  CAS  PubMed  Google Scholar 

  68. Sheehan B, Nigrovic LE, Dayan PS, Kuppermann N, Ballard DW, Alessandrini E, et al. Informing the design of clinical decision support services for evaluation of children with minor blunt head trauma in the emergency department: a sociotechnical analysis. J Biomed Inform. 2013;46(5):905–13.

    Article  PubMed  Google Scholar 

  69. Strohm L, Hehakaya C, Ranschaert ER, Boon WP, Moors EH. Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors. Eur Radiol. 2020;30(10):5525–32.

    Article  PubMed  PubMed Central  Google Scholar 

  70. van der Stap L, de Heij AH, van der Heide A, Reyners AK, van der Linden YM. Clinical decision support system to optimise symptom management in palliative medicine: focus group study. BMJ Support Palliat Care. 2021. https://0-doi-org.brum.beds.ac.uk/10.1136/bmjspcare-2021-002940.

  71. Vandenberg AE, Vaughan CP, Stevens M, Hastings SN, Powers J, Markland A, et al. Improving geriatric prescribing in the ED: a qualitative study of facilitators and barriers to clinical decision support tool use. Int J Qual Health Care. 2017;29(1):117–23.

    PubMed  Google Scholar 

  72. Weber S, Crago EA, Sherwood PR, Smith T. Practitioner approaches to the integration of clinical decision support system technology in critical care. J Nurs Adm. 2009;39(11):465–9.

    Article  PubMed  Google Scholar 

  73. Yılmaz AA, Ozdemir L. Development and Implementation of the clinical decision support system for patients with cancer and nurses’ experiences regarding the system. Int J Nurs Knowl. 2017;28(1):4–12.

    Article  PubMed  Google Scholar 

  74. Zaidi STR, Marriott JL. Barriers and facilitators to adoption of a web-based antibiotic decision support system. South Med Rev. 2012;5(2):42–9.

    PubMed  PubMed Central  Google Scholar 

  75. Zaidi STR, Thursky KA. Using formative evaluation to improve uptake of a web-based tool to support antimicrobial stewardship. J Clin Pharm Ther. 2013;38(6):490–7.

    Article  CAS  PubMed  Google Scholar 

  76. Gama F, Tyskbo D, Nygren J, Barlow J, Reed J, Svedberg P. Implementation frameworks for artificial intelligence translation into health care practice: scoping review. J Med Internet Res. 2022;24(1):e32215.

    Article  PubMed  PubMed Central  Google Scholar 

  77. Ronan CE, Crable EL, Drainoni ML, Walkey AJ. The impact of clinical decision support systems on provider behavior in the inpatient setting: a systematic review and meta-analysis. J Hosp Med. 2022;17(5):368–83.

    Article  PubMed  Google Scholar 

  78. Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implement Sci. 2013;8(1):139.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

The authors would like to thank Hannah Carter, Sanjeewa Kularatna, and Nicole White for their assistance with preliminary title and abstract screening. The authors gratefully thank the Queensland University of Technology for supporting this work.

Funding

This work was supported by funding from the Digital Health Cooperative Research Centre (DHCRC-0058) and SM is supported by a National Health and Medical Research Council administered fellowship (#1181138). The funders had no role in study design or decision to submit for publication.

Author information

Author notes

  1. Bridget Abell and Sundresan Naicker have equal contribution as co-first authors.

Authors and Affiliations

  1. Australian Centre for Health Services Innovation and Centre for Healthcare Transformation, School of Public Health and Social Work, Faculty of Health, Queensland University of Technology, Brisbane, QLD, Australia

    Bridget Abell, Sundresan Naicker, David Rodwell, Thomasina Donovan, Amina Tariq, Robin Blythe, Rex Parsons & Steven M. McPhail

  2. Biomedical Informatics and Digital Health, School of Medical Sciences, Faculty of Medicine and Health, The University of Sydney, Camperdown, Australia

    Melissa Baysari

Authors
  1. Bridget Abell
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sundresan Naicker
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. David Rodwell
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Thomasina Donovan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Amina Tariq
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Melissa Baysari
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Robin Blythe
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Rex Parsons
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Steven M. McPhail
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

BA and SM conceived the review. BA conducted the systematic search. DR, TD, SN, BA, RP, RB, AT, and MB conducted the screening. DR, TD, SN, and BA conducted the data extraction and analysis. DR, TD, SN, BA, RB, RP, AT, MB, and SM drafted and edited the manuscript.

Corresponding author

Correspondence to Sundresan Naicker.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

Refinement of search strategy.

Additional file 2.

Visual summary of synthesis and analysis process and mapping to key results and figures.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Abell, B., Naicker, S., Rodwell, D. et al. Identifying barriers and facilitators to successful implementation of computerized clinical decision support systems in hospitals: a NASSS framework-informed scoping review. Implementation Sci 18, 32 (2023). https://0-doi-org.brum.beds.ac.uk/10.1186/s13012-023-01287-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://0-doi-org.brum.beds.ac.uk/10.1186/s13012-023-01287-y

Keywords

Implementation Science

ISSN: 1748-5908