Background: Monitoring hand hygiene compliance (HHC) of healthcare providers (HCPs) in healthcare facilities is critical for hand hygiene (HH) promotion. However, less is known about the cost and effectiveness of different HHC monitoring tools. In this study, we aimed to compare various health economic indicators corresponding to electronic system-based monitoring (ESM) and manual paper-based monitoring (MPM) for HHC to provide evidence-based advice for HHC monitoring measures targeted selecting. 

Methods: A before and after self-controlled prospective study in 40 clinical departments with 4,524 HCPs was conducted from November 2022 to January 2023 (MPM implementation phase) and March 2023 to May 2023 (ESM implementation phase). The cost-effectiveness, cost-efficiency, the extent of the Hawthorne effect, and indirect cost–benefit of the two monitoring methods were compared. 

Results: In high- and medium-risk clinical departments, the cost-effectiveness ratio of ESM (7,977.90 CNY and 13,794.60 CNY) was lower than that corresponding to MPM (9,039.61 CNY and 14,549.05 CNY). Compared with ESM, every 1% increase in the HHC of MPM resulted in a 39,881.62 CNY reduction in cost. Acceptability of ESM was higher when the cost was 40,000 CNY or less. The cost-efficiency ratio of MPM in all departments (155,775.56 CNY) was higher than that of ESM (36,796.76 CNY). The Hawthorne effect of MPM of HHC in all departments (43.99%) was more pronounced than that of ESM (35.69%), with a statistically significant difference between the two monitoring methods (p < 0.01). When ESM was used, the rate of healthcare-associated infections (1.39%) in all departments was higher than the case when MPM was used (1.34%), but no statistical difference was noted (p = 0.562). 

Conclusions: For high-risk departments with multiple HH moments and more requirements for HHC monitoring frequency and coverage, ESM exhibited notable advantages over MPM in terms of cost-effectiveness, cost-efficiency, cost–benefit, and the Hawthorne effect.

Introduction

Healthcare-associated infections (HAIs) have become a globally recognized public health issue. The occurrence of HAIs, on the one hand, affects the recovery of patients, prolongs hospital stays, increases patients’ financial burden, and poses a threat to their physical health and life safety, and on the other, it leads to a higher incidence of medical disputes, reduces hospital bed turnover rates, and wastes a significant amount of healthcare resources [1]. Hand hygiene (HH) is the most basic, direct, affordable, and effective method for reducing HAIs [2]. The global hand hygiene compliance (HHC) rate among HCPs in healthcare facilities is reportedly suboptimal, ranging from 20% to 40%, which falls significantly short of the HH standards (90%–95%) for HCPs during medical activities established by the World Health Organization (WHO). The first global strategy for infection prevention and control (IPC) placed special emphasis on the requirement for continual surveillance of HH indicators to facilitate prompt feedback [3]. Additionally, it recommended that monitoring the HHC of HCPs in healthcare facilities is a crucial element of HH promotion programs [4]. The inconsistent distribution of regional healthcare resources globally has resulted in varying risks of HAIs among different disease groups within healthcare facilities. A critical issue when it comes to monitoring HH is selecting the most efficient monitoring tool for a particular situation, taking into account the available workforce, healthcare resources, acceptable HAI rates, and cost-effectiveness. 

Currently, the most widely accepted approaches for HH monitoring around the globe are direct observation and indirect observation [5]. Direct observation, that is, “trained observers directly observing HHC” is still regarded as the “gold standard” for HHC monitoring and is the one of the most reliable measures of evaluating HHC [6-7]. This monitoring method is fairly straightforward and can be implemented regardless of the size or existing structure of the hospital. However, it is prone to the Hawthorne effect and is overly tedious and time-consuming, making it difficult to obtain accurate results. Also, this approach makes it challenging to conduct large-scale surveys [8]. Indirect monitoring includes but is not limited to estimating the consumption of paper towels, hand sanitizers, and soap, among other products, for maintaining HH or assessing the needs based on the nursing operation database. These methods require less time and resources compared with direct observation but have some biases, such as a lack of evaluation of patient factors and workload. In recent years, with the continuous development of technology, intelligent monitoring systems have been widely used in healthcare facilities and are considered potential alternative solutions for measuring and improving HHC. Previous studies documented the usage of electronic badges with alcohol vapor sensors to monitor whether HCPs practiced HH [9]. Some researchers have used smart rings worn on the fingers to monitor HHC. Compared with the most common manual paper-based direct observation method, using information systems for HHC monitoring has prominent advantages such as sustainability, reduced Hawthorne effect, saved human resources, improved data collection efficiency, and increased data traceability. Further investigations should be performed to evaluate the practicality, cost, acceptability, and cost-effectiveness of HHC monitoring information systems, even though they provide a novel way of accurately monitoring HHC.

This study was conducted to compare various health economic indicators of electronic system monitoring (ESM) and manual paper-based monitoring (MPM) for HHC and to utilize the results to furnish healthcare institutions with varying levels of resources and infection risk with HHC monitoring evidence-based advice.

Study design

A before and after self-controlled prospective study in 40 clinical departments with 4,524 HCPs of Zhongnan Hospital of Wuhan University, China, was conducted from November 2022 to January 2023 (MPM implementation phase) and March 2023 to May 2023 (ESM implementation phase; Figure 1). MPM analysis relied on handwritten HHC records, with data being analyzed manually. ESM was accomplished using a self-developed app (patent number: 202230575169.5) embedded in nursing operation mobile including the following functions: HHC monitoring, automatic statistics, inquiry, analysis, reminders, and so on. This study has been approved by the Clinical Research Ethics Committee of Zhongnan Hospital of Wuhan University (approval number: 2023136K).

Figure 1: Flow chart comparing MPM and ESM.

The cost-effectiveness, cost-efficiency, Hawthorne effect, indirect cost–benefit analysis of the two monitoring methods were compared. 

Basic data

Basic data included information concerning beds and HCPs of 40 clinical departments. In addition, these 40 clinical departments were divided into three levels of high risk (11 clinical departments), medium risk (23 clinical departments), and low risk (six clinical departments) according to the previous literature [11-13] (s table 1).

Table 1: Classification of departments

Costs

The cost of ESM included the cost of software, electricity, and labor cost.  ① the cost of developing software (C1): the development cost of the software was 50,000 CNY(1 CNY equals about $0.14 [17]), the use cycle is at least ten years; our study lasted three months, and the average payment per department was 31.25 CNY; ② the cost of electricity (C2): a full charge required one degree of electricity, and according to the power supply standard of the hospital, the cost of a full charge of the device was 0.8 CNY [14], where the charging time of device was once a month; ③ labor cost (C3): According to the investigation and statistics of Wuhan Municipal Bureau of Human Resources and Social Security, the average monthly salary of employees in 2022 was 8,845 CNY [15], and the hourly labor cost was about 40 CNY. The APP for monitoring HHC used in this study was nested on the original electronic equipment of the hospital as an additional function. During the use of the device for ESM of HH, there was no phenomenon of equipment damage during the monitoring and after use. Therefore, the potential maintenance cost caused by equipment damage was not attributed to the monitoring of HHC in this study.

The cost of MPM included the cost of paper and the labor cost. ① cost of paper: the cost of an A4 paper was estimated to be 0.04 CNY. According to the design of WHO HHC questionnaire [16], it took two sheets of paper to conduct an MPM; ② labor cost (C3). The costs and calculation of the two monitoring methods are summarized in Table 1.

Table 1: Cost calculation of two monitoring methods

HHC Monitoring Effectiveness 

HHC includes hand wash and alcohol-based hand rub. HHC means that HPWs adhere to the established HH guidelines and the steps, time, and scope of HH are in line with regulations. HHC cost-effectiveness was quantified by monitoring HH action and opportunity by respective part-time IPCs from clinical departments and based on equation 1 [18]:

HHC monitoring effectiveness = (HH action)/ (HH opportunity) ×100% (equation 1)

HHC Monitoring Efficiency

Cost-efficiency was evaluated by time consumed on HHC monitoring and investigated by 10 full-time IPCs from departments of IPC and nursing. HHC was quantified through ESM by directly inputting the observed number of HH actions, and the system automatically generated the result of HHC and uploaded the same to the computer. The use of MPM required the observers to record the observed and actual HH moments in a paper observation sheet and calculate the results manually. HHC monitoring efficiency (equation 2) was estimated as the time spent for each monitoring moment:

HHC monitoring efficiency = 

×100% (equation 2)

Hawthorne effect

The Hawthorne effect refers to a change in behavior as a motivational response to the interest, care, or attention received through observation and assessment [19]. The Hawthorne effect may deliberately increase HH when the observer perceives that he/she is being observed during HH monitoring. In this study, the HHC during the period when HCPs were not observed, i.e., the unobservation period of different risk departments was derived from previous references (sTable 2). The extent of the Hawthorne effect was calculated using equation 3 [20]:

Hawthorne effect = 

×100% (equation 3)

Table 2: The HHC during no observation of different risk departments

Indirect benefit

Indirect benefit was evaluated in terms of disease burden due to HAIs during the monitoring period. HAI rates were estimated as the number of new infections in the total number of hospitalized patients in a certain period of time (equation 4):

HAI rates = 

×100% (equation 4)

Data on HAI rates in this study were exported from the hospital information system of Zhongnan Hospital of Wuhan University. HAI rates without HH of different risk departments were estimated from references (sTable 3). The average disease burden per case of HAIs was 39,800 CNY estimated from references [25]. 

Table 3: The HAIs rates without HH of different risk departments

Outcomes

Cost-effectiveness ratio, cost-efficiency ratio, Hawthorne effect, and indirect cost–benefit ratio are the main outcomes and target effects and based on equations 5 to 9:

Cost-effectiveness ratio = 

×100% (equation 5) ；

Cost-efficiency ratio = 

×100% (equation 6) ；

Cost–benefit ratio (CBR) = 

×100% (equation 7) ；

= 

×100% (equation 8):

Incremental cost-effectiveness ratio=

×100% (equation 9).

Data and Analyses

Cost calculation was recorded into Microsoft Excel by two research members, and the equation was set in Excel and calculated to avoid any computational errors. SPSS (version 16, SPSS Inc) software was used for statistical analysis, and the chi-square test was used to detect any statistical differences within and between groups. Based on the previous studies and combined with the data characteristics of this study, the decision tree model [10] was used to analyze cost-effectiveness. The decision node of the decision tree model was HHC monitoring, and the branches of the opportunity node were ESM and MPM. The nodes of these two branches are divided into HHC and its lack thereof, i.e., no-HHC. The decision tree model was performed using Tree Age Pro 2022 (Figure 2). 

Figure 2: Decision tree model.

Comparison of cost-effectiveness of the two monitoring methods among different departments

The total cost spent on ESM for the 40 departments (17,702.92 CNY) was 4,123.76 CNY lower than that of MPM (21,826.68 CNY). The HHC of MPM (80.16%) was higher than that of ESM (69.82%), and the difference was statistically significant (p < 0>

In high- and medium-risk departments, the cost-effectiveness ratio of ESM (7,977.90 CNY and 13,794.60 CNY, respectively) was lower than that of MPM (9,039.61 CNY and 14,549.05 CNY, respectively), indicating that the average cost of ESM was lower when improving HHC. On the contrary, in low-risk departments, the cost-effectiveness ratio of ESM (3,910.77 CNY) was higher than that of MPM (3,899.06 CNY), indicating that the average cost of ESM was higher. Compared with ESM, the incremental cost of MPM in all departments was 4,123.76 CNY, the incremental effectiveness was 10.34%, and the incremental cost-effectiveness ratio was 39,881.62 CNY. Every 1% increase in the HHC compliance of MPM was found to result in a 39,881.62 CNY reduction in cost. However, in low-risk departments, every 1% increase in the HHC of MPM was noted to result in a 3,828.29 CNY reduction in cost. As shown in Figure 3, both ESM and MPM are undominated strategies. Our tornado analysis revealed that the cost of MPM was the only factor that had the largest impact of the overall strategy (Figure 4).

Figure 3: Cost-effectiveness analysis of using the two monitoring methods among the different risk departments Undominated (An undominated strategy indicates that when there are multiple strategies to choose from, the dominant strategy is better than the others).

Figure 4: One-way sensitivity analysis on incremental cost.

Table 2: Cost-effectiveness analysis and incremental cost-effectiveness analysis of the two monitoring methods among the different risk departments.

The cost-effectiveness acceptability curve (CEAC, Figure 5) illustrates that when the cost of all departments, namely, high-risk departments, medium-risk departments, and low-risk departments was 40,000 CNY, 15,000 CNY to 20,000 CNY and 5,000 CNY to 10,000 CNY, respectively, the choice of ESM had a higher cost-effectiveness.

Figure 5: Cost-effectiveness acceptability curve of using the two monitoring methods among the different risk departments.

Comparison of cost-efficiency analysis of using two monitoring methods among different departments

Between the two monitoring methods, the efficiency of ESM (48.11%) in all departments was higher than that of MPM (14.20%), and the difference was statistically significant (p < 0>

departments (155,775.56 CNY) was higher than that of ESM (36,796.76 CNY), indicating that ESM was efficient and low-cost. In high-risk departments, the maximum gap in the cost-efficiency ratio of the MPM and ESM were 72,013.35 CNY and 8,858.02 CNY, respectively (Table 3).

Table 3: Cost efficiency analysis of using two monitoring methods among the different risk departments.

Comparison of the extent of Hawthorne effect of using two monitoring methods among different departments

The extent of Hawthorne effect of MPM of HHC in all departments (43.99%) was higher than that of ESM (35.69%), and the difference between the two monitoring methods was noted to be statistically significant (p < 0>

of MPM (59.45%) and ESM (47.90%) in high-risk departments were all higher than those in other departments, and no statistical difference existed between the two monitoring methods in high-risk departments (p = 0.940). The extent of Hawthorne effect of MPM (41.46%) and ESM (36.20%) in medium-risk departments was the lowest, and there existed a statistical difference between the two monitoring methods (p < 0>

Table 4. Hawthorne effect analysis of using the two monitoring methods among different departments.

Comparison of cost-benefit analysis of the two monitoring methods among different departments

When ESM was used as the HHC monitoring approach, the HAI rates (1.39%) in all departments were higher than that when MPM was used (1.34%), but no statistical difference was observed (p = 0.562). When comparing the CBR of MPM and ESM, the average CBR of ESM in all departments (665.44 CNY) was lower than that of MPM (2,454.37 CNY), 

indicating that in the current analysis, the CBR of ESM was better than MPM. Among departments with different risk levels, the CBR difference between the two monitoring methods in medium-risk departments was found to be the greatest. The average CBR of ESM (744.73 CNY) was 2,560.72 CNY lower than that corresponding to MPM (3,305.45 CNY; p < 0>

Table 5: Cost-benefit analysis of using the two monitoring methods among the different risk departments.

The WHO Hand Hygiene Research Agenda for 2023–2030 underscores the need to advance HH research and increase the effectiveness of HHC monitoring through the use of information technology in the next two decades [30]. HHC monitoring is widely recognized as a highly challenging task that requires expertise and training, as this determines the accuracy of the monitoring results, and it also demands human resources and time investment, as this determines the completeness of the monitoring and the timeliness of feedback. This study marked the initial examination of a comprehensive health economic analysis to evaluate the utilization of ESM and MPM in HHC monitoring. The findings revealed that ESM outperformed MPM in terms of cost-effectiveness, cost-efficiency, cost–benefit, and the extent of Hawthorne effect.

In terms of cost-effectiveness, the total cost of ESM was lower than that of MPM. Particularly in high- and medium-risk departments, the average cost 

of improving HHC using ESM was lower than that of MPM. In line with this study is the observation that most commonly used information technology systems for HHC motoring can significantly improve HHC among HCPs (OR = 3.06, p < 0>

the cost of installation [34]. Hence, MPM were recommended to be applied in low-risk medical departments. 

As for cost-efficiency, ESM was noted to be superior to MPM, especially in high- and medium-risk departments, where their efficiency advantage was even more pronounced. We theorized that this could be attributed to the higher-risk divisions, such as the intensive care unit, executing more intrusive procedures, accommodating patients with weakened immune systems, and consuming a great number of resources in the hospital. The staff in this department were short-handed (with a nurse-to-patient ratio of 2.5 to 3:1 [35]), and there were shortages of equipment, drugs, and other resources, all of which put an extra strain on HHC, making it necessary to keep a close eye on personnel and time expenditure. ESM could help managers better understand the implementation, promptly identify problems and non-compliant behavior, and avoid waste and misuse of resources in high-risk departments. Moreover, through analysis of HH data, resource allocation could be optimized, and work efficiency can be improved [36]. This study’s analysis of cost-efficiency is consistent with the actual clinical demands. ESM could simultaneously monitor the HHC of multiple individuals online, and the data could be automatically analyzed, provided with feedback, and traced, thereby saving a significant amount of workforce and time costs. Kardaś-Słoma [9] also remarked that ESM can automatically collect data, improving the accuracy and timeliness of monitoring. The data collected through an ESM can be further analyzed and used for trend prediction. This analysis can help identify hotspots and high-risk periods for HH issues, enabling targeted preventive measures to improve HHC and correctness. 

The Hawthorne effect, which refers to the alteration of behavior on being aware that one is being observed, was chosen as an evaluation indicator for monitoring biases. ESM, being discreetly conducted on nursing mobile devices, offers a certain level of secrecy when conducting compliance monitoring. On the other hand, MPM relies on paper records, making the Hawthorne effect more pronounced. Therefore, ESM was more effective in controlling the Hawthorne effect. Interestingly, it was found that HHC was lower under ESM compared with that under MPM, which may be attributed to a weaker Hawthorne effect associated with ESM. Casaroto’s research [36] in the ICU context demonstrated an HHC rate of 56.3% under MPM and 51.0% under ESM, with the Hawthorne effect being the influencing factor. Another research group [5] demonstrated a downward trend in HHC rates among HCPs after the introduction of intelligent monitoring systems. This phenomenon mainly occurs when direct observation and ESM are used simultaneously to measure HHC. Direct observation produces the Hawthorne effect, resulting in higher measurement results compared with those obtained through intelligent monitoring systems. These findings suggest that information technology-enabled monitoring provides a more accurate reflection of the actual situation, enabling the establishment of more precise baseline data for subsequent HHC interventions. Electronic HHC motoring systems can monitor HHC on all work shifts without a Hawthorne effect and provide significantly more data regarding HHC [4]. 

Finally, a comparison of the occurrence of HAIs during the application of the two monitoring methods was conducted. Although no significant difference was observed in the occurrence of HAIs between ESM and MPM during the study period, the cost-effectiveness of ESM outweighed that of MPM when considering the cost and the burden of patient diseases caused by HAIs. Likewise, Salinas-Escudero [37] observed that within one month of implementing ESM, the number of infections decreased by 46–79 individuals, resulting in cost savings of $308,927 to $546,795 for preventing HAIs. 

However, it is important to note some limitations of this study. First, a large number of studies focused on using electronic monitoring systems to monitor HHC, including application-assisted direct observation, camera-assisted observation, sensor-assisted observation, and real-time locating system [4]. We cannot evaluate the health economics of all electronic monitoring systems. Hence, the findings cannot be extrapolated to other electronic monitoring systems. However, we intend to conduct an exploratory health economic evaluation in terms of tools for monitoring HHC and provide a reference for future research. Second, the cost of ESM did not consider the potential repair costs resulting from device damage in the later stages. Long-term prospective observational studies are also being conducted to provide a comprehensive health economic assessment of the long-term application of ESM in HHC. 

In conclusion, for high-risk departments with multiple HH moments and higher requirements for HHC monitoring frequency and coverage, ESM exhibits notable advantages over MPM in terms of cost-effectiveness, cost-efficiency, cost–benefit, and the Hawthorne effect.