Evaluating the outcomes of patient safety education programs in nursing education: a scoping review

Abstract

Purpose

This study aimed to identify the trends in scholarly research related to patient safety education programs and outcome evaluations in the field of nursing education, focusing on providing a comprehensive overview of the current state of patient safety education and suggesting directions for future program development and evaluation.

Methods

We conducted a scoping review using Arksey and O’Malley’s 5-step framework and searched five databases; Cumulative Index to Nursing and Allied Health Literature (CINAHL), Education Resources Information Center (ERIC), PubMed, Science Direct, and Research Information Sharing Service (RISS) for Korean literature.

Results

Of the 1626 articles retrieved, we identified 27 articles published between 2006 and 2023. The articles’ educational content focused on comprehensive patient safety education using instructional methods such as lectures, discussions, simulations, and online education. The effectiveness of the programs was evaluated primarily via knowledge, skills, and attitudes toward patient safety, and behavioral and results-related outcomes were rarely evaluated at Kirkpatrick’s evaluation., The most commonly used tools were the Health Professional Education in Patient Safety Survey and the questionnaire developed by the World Health Organization Patient Safety Program. The evaluation was conducted using quantitative and qualitative methods, but follow-up tests were rarely conducted.

Conclusion

Our review findings suggest that patient safety education programs for nursing students and clinical nurses should apply diverse teaching methods and comprehensive evaluation strategies to enhance their overall effectiveness and ensure the practical application of safety practices. It is essential to diversify program evaluation metrics, such as experiential learning and interprofessional education, by expanding performance indicators at both learner and institutional levels for long-term repeated assessments.

Background

With advancements in healthcare standards, provision of safe care and prevention of harm to patients have become paramount issues. Nurses play a pivotal role in patient safety by ensuring effective communication, collaborating with other healthcare professionals for holistic care, educating patients and families, advocating for patient needs, implementing safety protocols, and fostering a safety culture by encouraging open communication about errors and near misses [1, 2]. These actions significantly contribute to reducing harm to patients and enhancing overall care quality.

Comprehensive educational programs are essential to ensure that nurses have appropriate competencies to maintain patient safety and adhere to established safety protocols. A higher level of understanding and positive attitude toward patient safety are associated with better compliance with safety protocols [3,4,5], which in turn reduces the incidence of adverse events and improves overall patient outcomes. Therefore, it is crucial to assess and enhance nurses’ awareness and attitudes toward patient safety through targeted education and training programs to ensure the highest standards of patient care [6]. Accordingly, the World Health Organization (WHO) has developed frameworks emphasizing the importance of building healthcare worker competencies to improve patient safety [7], and various educational programs on patient safety are being actively developed and implemented [8].

Evaluation is the process of judging learners’ learning outcomes, clinical performance, competencies, and educational programs based on assessments [9]. In nursing education, evaluations focus on the knowledge acquired by participants, educational outcomes, and the quality of nursing activities performed in clinical settings. Outcome evaluation of educational programs is critical for understanding their effectiveness and ensuring that they meet the required standards. Additionally, proper evaluation helps in identifying the strengths and weaknesses of educational programs, thereby guiding future development and enhancing educational outcomes [10].

Evaluating the outcomes of patient safety education programs can provide valuable insights and guide the development of future educational initiatives, ensuring that they are more effective and impactful [3]. Despite its recognized importance, research specifically focusing on the outcome evaluation of patient safety education programs within the context of nursing education is limited. Previous reviews [3, 6] did not fully incorporate cross-sectional designs that evaluate outcomes following the implementation of educational curricula. The current scoping review addresses these limitations by integrating both intervention studies and cross-sectional research to provide a broader and more nuanced understanding of the effectiveness of patient safety education programs in nursing education. Additionally, there is a need for studies that propose approaches and strategies for educators to use in evaluating these programs by analyzing existing research and identifying gaps [11].

Objectives

This scoping review aimed to identify trends in scholarly research related to patient safety education programs and outcome evaluations in the field of nursing education. By focusing on outcome evaluations, this review seeks to provide a comprehensive overview of the current state of patient safety education for clinical nurses and nursing students, highlight areas requiring further research, and suggest directions for future program development and evaluation.

Methods

Study design

This scoping review was carried out using Preferred Reporting Items for Systematic reviews and Meta-Analysis extension for Scoping Reviews (PRISMA-ScR) checklist [12], and was conducted following the five-stage framework of the scoping review methodology proposed by Arksey and O’Malley [13].

Study subject

The study subject were scholarly articles that have implemented and verified the effectiveness of patient safety education for both nursing students and clinical nurses. Reflecting on the characteristics of patient safety education that spans from undergraduate nursing education to the continuing education of clinical nurses, this study included articles that involve nursing students as well as those involving clinical nurses.

Study procedure

Stage 1: Identifying the research questions

The following research questions were established to analyze the current status of the effectiveness evaluation of patient safety education for clinical nurses and nursing students:

• (1) What are the trends in scholarly research on patient safety education programs for clinical nurses and nursing students?

• (2) What variables have been used to evaluate the effectiveness of patient safety education programs for clinical nurses and nursing students?

• (3) What methods are effective in patient safety education programmes for clinical nurses and nursing students?

Stage 2: Identifying relevant studies

A literature search was conducted from July 1 to July 10, 2023. Three researchers independently conducted the literature search. The search period was set with no restrictions on the starting date and included studies published up to June 30, 2023. The databases used for the search included Cumulative Index to Nursing and Allied Health Literature (CINAHL), Education Resources Information Center (ERIC), PubMed, and Science Direct. Additionally, for Korean literature, the Research Information Sharing Service (RISS) was included. The search terms “patient safety” and “nursing education” were combined using the AND operator.

Stage 3: Study selection

The literature search and selection processes were carried out according to the PRISMA-ScR guidelines (Fig. 1). The search results yielded 208 articles from CINAHL, 12 from ERIC, 1,222 from PubMed, 162 from Science Direct, and 22 from RISS, totaling 1,626 articles. The literature was managed using Endnote, which facilitated the organization of search results and the removal of 151 duplicates. After removing duplicates, the titles and abstracts of 1,475 articles were independently reviewed by three researchers at each stage. Discrepancies in the selection results were resolved by comparing individual results and conducting research meetings until a consensus was reached. The inclusion criteria were as follows: (1) studies that implemented and verified the effectiveness of patient safety education programs for clinical nurses and nursing students and (2) articles written in English or Korean. The exclusion criteria were as follows: (1) evaluation studies of the effectiveness of educational interventions without an explicit focus on patient safety and (2) conference abstracts, posters, books, and dissertations.

Fig. 1

PRISMA flow diagram of study screening. CINAHL = Cumulative Index to Nursing and Allied Health Literature; ERIC = Education Resources Information Center; RISS = Research Information Sharing Service

Of the 1,475 articles, after removing duplicates, 1,424 studies were excluded through a review of titles and abstracts. A total of 51 full-text articles were then reviewed. Among these, 15 studies were excluded because they evaluated the overall nursing education programs or nursing education systems, such as dedicated education units, rather than the effectiveness of patient safety education. Four studies were excluded because they did not include content related to patient safety in their educational programs. Three studies were excluded because they targeted patients, nurse practitioners, or nursing instructors, instead of clinical nurses or nursing students. In addition, two studies were excluded because the content of the educational programs was not clearly described. No documents corresponding to conference abstracts, posters, books, or dissertations were included during the full-text screening process. Therefore, a total of 24 studies were excluded from the analysis. Ultimately, 27 studies were selected for analysis (Appendix 1).

Stage 4: Charting the data

Data from the 27 selected studies were analyzed using Microsoft Excel. Three researchers independently recorded basic information related to the studies, including the author, year, country of publication, study design, educational intervention, participants, and evaluation indicators, timing, and tools and methods. The results recorded by the researchers were compared and any discrepancies were resolved through discussion to finalize the data. To evaluate the effectiveness of the educational programs, the variables used in the evaluations were analyzed according to Kirkpatrick’s four-level evaluation model [14], which includes reaction, learning, behavior, and result evaluations.

Stage 5: Collating, summarizing, and reporting the results

The collected data were descriptively analyzed using frequencies to summarize the studies’ general characteristics. The analysis focused on patient safety education interventions and effectiveness evaluation of patient safety education. Outcome variables were categorized and analyzed based on Kirkpatrick’s four-level evaluation model. Three researchers independently reviewed the summarized data, and discrepancies were resolved through consensus meetings. The research team collaboratively derived the conclusions and presented them using tables for clarity and precision.

Results

Characteristics of selected studies

The characteristics of the 27 studies selected for analysis are summarized in Table 1. The publication year of these studies ranged from 2006 to 2023. The studies were conducted in various regions, including South Korea [S17, S19, S20, S23, S25, S27], Iran [S15, S26], Japan [S10], Singapore [S14], Saudi Arabia [S22], and Taiwan [S4], encompassing a total of 12 studies from Asia; nine studies were from the United States [S2, S3, S5, S6, S7, S8, S11, S13, S16]; three studies were from Europe, specifically the United Kingdom [S1, S9] and Spain [S24]; two studies were from Oceania, specifically Australia [S18, S21]; and one was from Mozambique in Africa [S12]. The research design included 21 experimental studies [S2, S3, S4, S5, S6, S10, S11, S12, S14, S15, S16, S17, S18, S19, S20, S22, S23, S24, S25, S26, S27] and 6 nonexperimental studies [S1, S7, S8, S9, S13, S21].

Table 1 Characteristics of Selected Studies

The majority of the studies included nursing students as participants [S1, S2, S3, S4, S5, S6, S7, S8, S9, S10, S11, S14, S16, S17, S18, S19, S20, S24, S25, S26, S27], while eight studies included nurses [S6, S10, S12, S13, S15, S21, S22, S23]. Two studies [S6, S10] included both nurses and nursing students. One study included nurses and nursing managers [S15]. Additional studies included healthcare workers [S21], healthcare students [S20], or a combination of these groups [S6, S16]. The number of participants ranged from 19 to 429.

Patient safety education interventions

Educational content

Thirteen studies [S3, S6, S9, S13, S14, S15, S18, S19, S20, S22, S25, S26, S27] focused on comprehensive patient safety education, covering concepts, necessity, related factors, accident prevention, and management. Eight studies [S2, S6, S7, S11, S16, S17, S21, S23] emphasized patient safety risk factors in clinical settings. Specific patient safety topics, such as speaking up [S8], root cause analysis [S24], medication safety [S1, S4, S10, S12], fall prevention [S5], and transfusion safety [S14], were also addressed.

Educational methods

The educational methods used varied among the studies. Ten studies employed a single method [S2, S7, S8, S11, S14, S17, S18, S21, S22, S23], twelve studies used two methods [S1, S3, S4, S5, S9, S10, S12, S13, S15, S16, S24, S25], and five studies used three methods [S6, S19, S20, S26, S27]. The most commonly used method was lectures [18 studies; S1, S3, S4, S5, S6, S9, S10, S12, S13, S15, S18, S19, S20, S22, S24, S25, S26, S27, followed by discussions [9 studies; S2, S9, S13, S15, S17, S19, S20, S26, S27], simulations [8 studies; S2, S5, S6, S7, S11, S16, S21, S23], and online education [4 studies; S1, S4, S19, S25]. Clinical practicum [S6, S8, S21, S24], interprofessional education [S16, S20], games [S12, S14], and role-playing [S26, S27] were also used.

Evaluation of patient safety education effectiveness

Outcome variables and evaluation time point

The effectiveness of patient safety education was evaluated using various variables classified into four levels of the Kirkpatrick’s model. Reaction outcomes, such as learner satisfaction and effectiveness, were evaluated in nine studies [S4, S6, S12, S14, S20, S21, S22, S25, S27]. Learning outcomes were assessed in terms of knowledge, skills, attitudes, confidence or comfort levels, patient safety culture [S15], critical thinking [S6, S23], clinical decision-making [S6], and problem-solving abilities [S23]. Behavioral outcomes were evaluated in two studies by measuring medication error rates [S1] and expected patient safety activities through safety rounding [S21]. Results-related outcomes, such as the impact of education on organizational or patient outcomes, were evaluated in two studies by analyzing the annual patient safety incident rates [S6, S21].

Except for one study [S12], all studies measured outcomes before and after the educational intervention, with one study conducting a follow-up test three months post-intervention [S26].

Evaluation methods and tools

Methods of evaluating the effectiveness of educational interventions included both quantitative and qualitative approaches. All studies employed quantitative methods, with all but one [S1] using questionnaires. Eleven studies [S2, S3, S4, S5, S6, S11, S12, S13, S14, S16, S20] used self-developed questionnaires, while others used established tools, such as the Health Professional Education in Patient Safety Survey [S8, S18, S26, S27], WHO Patient Safety Program Questionnaire [S9, S27], and Patient Safety Competency Self-Evaluation [S19, S25].

Measurement instruments reported in the studies and frequency found in the literature are presented in Table 2. The most commonly used instruments were the Health Professional Education in Patient Safety Survey (Ginsburg et al., 2010) [S8, S26], Health Professional Education in Patient Safety Survey (Ginsburg et al., 2012) [S18, S27] and the questionnaire developed by the WHO Patient Safety Program [S9, S27]. The Health Professional Education in Patient Safety Survey (Ginsburg et al., 2010) comprises 38 items rated on a 5-point Likert scale, and is divided into three sections: students’ confidence in what they learned about keeping patients safe, health professional education, and comfort in speaking up for patient safety. The Health Professional Education in Patient Safety Survey (Ginsburg et al., 2012) comprises 16 items rated on a 5-point Likert scale, and is divided into six sections: working in teams, communicating effectively, managing safety risks, understanding human and environmental factors, recognizing and responding to adverse events, and culture of safety. The questionnaire, developed by the WHO Patient Safety Program comprises 28 items divided into two categories: attitudes toward understanding patient safety (a Likert type scale) and evaluating knowledge (multiple-choice questions).

Table 2 Measurement Instrument Reported in the Studies and Requency Found in the Literature

Six studies [S1, S6, S11, S14, S16, S17] included direct observations with quantitative analysis, and two studies [S6, S21] analyzed patient safety incident statistics. Six studies [S3, S5, S12, S20, S21, S25] applied both qualitative and quantitative methods, such as focus group discussions [S3], analysis of reflective journals [S5], written feedback [S12, S25], and participant interviews [S12, S20, S21].

Discussion

This review aims to identify trends in scholarly research related to patient safety education programs and outcome evaluations in nursing education. Focusing on outcome evaluations, it sought to provide a comprehensive understanding of the current state of patient safety education for nursing education and propose recommendations for future program development and evaluation.

Improvement of evaluation methods for experiential learning and interprofessional education

The evolving complexities of clinical settings require a broader spectrum of experiential learning strategies [15]. Simulations, games, and role-playing identified in this review can effectively link theoretical learning with practical skills [16,17,18]. These methods not only facilitate experiential learning, but also enhance the engagement and retention of critical safety practices among learners [19]. Future educational programs should consider a blended learning approach to maximize experiential learning, thereby enhancing patient safety in healthcare settings.

Furthermore, in highly complex healthcare environments, the involvement of various healthcare providers in multiple procedures underscores the importance of communication and teamwork in patient safety [20, 21]. In the studies analyzed in this review, the importance of teamwork and collaboration in patient safety was highlighted through active team-based educational practices such as group discussions. However, only two studies included interprofessional education (IPE) involving clinical nurses or nursing students, indicating the need to expand interdisciplinary education. IPE can facilitate an understanding of team dynamics in clinical settings, enhance communication skills, and foster a shared commitment to patient-centered care [22]. This expansion can further enhance understanding and cooperation among different healthcare professionals, thereby improving patient safety outcomes [23].

The expansion of experiential learning and IPE underscores the need for valid and reliable methods of measuring educational outcomes. Many of the studies analyzed in this review measured perceptions of knowledge, skills, and attitudes related to patient safety using questionnaires [S3, S6, S7, S14, S19]. The measurement of educational outcomes should extend not only to learners’ perceptions of confidence and competency, but also to the systematic measurement of objectively demonstrated educational outcomes [24]. While traditional methods such as objective structured clinical examinations (OSCE) and clinical practice evaluations (CPE) are valuable, evaluating patient outcome data associated with educational interventions is crucial to align with results (Kirkpatrick’s Level 4) and demonstrate the real-world impact of training. To address challenges in implementing experiential learning and IPE, virtual environments and technological advancements, such as AI-driven tools like chatbots for communication training, can supplement traditional methods. These approaches enable experiential learning and IPE while enhancing both learner and patient-level outcome evaluations, reflecting current educational trends.

Need for expanding evaluation criteria and methods for educational outcomes

Most studies in this review assessed reactions and learning outcomes post-education, with only three studies evaluating behavior change and organizational-level outcomes, and only one study including follow-up measurements. Previous studies analyzing the effectiveness of patient safety education implemented in medical and pharmaceutical education have also found that very few studies have analyzed learners’ behavioral changes or their impact on institutions [25, 26]. However, it is important to acknowledge that patient safety outcomes are influenced by numerous variables beyond education, making it challenging to establish causality [31].

The results of this review suggest the necessity of two key aspects in the evaluation of patient safety education: long-term evaluations of educational effectiveness and the expansion of performance indicators to include both learner and institutional levels. Long-term repeated assessments are required to establish a feedback system that can continuously improve patient safety competencies [27]. Futhermore, evaluating the impact of education on institutions and patient safety requires specific measures. Learner-level outcomes include changes in knowledge, skills, attitudes, and behaviors. Institutional outcomes focus on patient safety metrics, such as reduced errors, improved adherence to protocols, staff retention, and safety culture. These measures provide insights into the effectiveness of nursing education and encourage institutional and governmental investment [28]. In conclusion, such evaluations are crucial for determining the actual impact of educational programs on clinical practices and patient outcomes, suggesting the need for systematic enhancement of training curricula and methodologies to better support ongoing professional development and patient safety enhancement. Furthermore, healthcare institutions and governments should not only invest in patient safety education but also ensure transparency by providing data on educational outcomes and reporting results.

Qualitative data serve as a crucial resource for analyzing students’ educational experiences and interactions in the evaluation of educational outcomes. Examining individual case studies helps identify differences among learners, and analyzing common cases provides a comprehensive evaluation of educational experiences. This holistic perspective on the context, process, and outcomes of educational programs facilitates the identification of areas for improvement [29]. During the research process, no qualitative studies specifically focused on the effectiveness of patient safety education. Among the six studies analyzed, qualitative data analysis was limited to the use of reflective journals, interviews and discussion content, and written feedback to triangulate the results derived from quantitative data. Therefore, it is essential to consider new approaches to enhance the understanding of educational outcomes and identify areas for program improvement [30]. Qualitative and qualitative-dominant mixed methods research can be conducted on topics such as clinical practicums or simulation experiences related to patient safety, ethnography on behavioral changes in learners after educational implementation, and grounded theory on learner interactions during program application. These diverse themes offer a comprehensive exploration of educational experiences and outcomes, and provide valuable insights for improving educational programs.

Limitations

This study reinforces the need to focus on analyzing the evaluation variables and methods of patient safety education programs implemented in nursing education, ultimately providing guidelines that can be utilized by nursing educators in the future. However, this study was limited in that it selected only studies that specifically set patient safety as an educational outcome and implemented interventions accordingly. Consequently, studies that included patient safety elements as part of the broader knowledge and skills training in nursing education were excluded from the analysis.

Conclusion

The importance of patient safety in healthcare settings mandates rigorous and continuous education for those on the frontline, including nurses and nursing students. Patient safety education in nursing is implemented through various methods, including simulations, and should be expanded to include interprofessional education to enhance teamwork and collaborative skills. Appropriate evaluation methods must be applied in this context. Additionally, it is crucial to establish long-term and systematic evaluation frameworks that extend beyond learners’ reactions and learning outcomes to include behavioral changes and institutional impacts. Utilizing qualitative data is essential to achieve a comprehensive understanding of educational outcomes.

Based on the results of this review, future research should explore various performance evaluations, including cost-effectiveness studies of educational outcomes, long-term, large-scale outcome assessments, and evaluations using qualitative analyses. Follow-up studies are expected to provide valuable insights into the effectiveness and impact of patient safety education programs.