567.full
Abstract
Background: Conflicting recommendations exist related to which facial protection should be used by health care workers to prevent transmission of acute respiratory infections, including pandemic influenza. We performed a systematic review of both clinical and surrogate exposure data comparing N95 respirators and surgical masks for the prevention of transmissible acute respiratory infections.
Methods: We searched various electronic databases and the grey literature for relevant studies published from January 1990 to December 2014. Randomized controlled trials (RCTs), cohort studies and case–control studies that included data on health care workers wearing N95 respirators and surgical masks to prevent acute respiratory infections were included in the meta-analysis. Surrogate exposure studies comparing N95 respirators and surgical masks using manikins or adult volunteers under simulated conditions were summarized separately. Outcomes from clinical studies were laboratory-confirmed respiratory infection, influenza-like illness and workplace absenteeism. Outcomes from surrogate exposure studies were filter penetration, face-seal leakage and total inward leakage.
Results: We identified 6 clinical studies (3 RCTs, 1 cohort study and 2 case–control studies) and 23 surrogate exposure studies. In the meta-analysis of the clinical studies, we found no significant difference between N95 respirators and surgical masks in associated risk of (a) laboratory-confirmed respiratory infection (RCTs: odds ratio [OR] 0.89, 95% confidence interval [CI] 0.64–1.24; cohort study: OR 0.43, 95% CI 0.03–6.41; case–control studies: OR 0.91, 95% CI 0.25–3.36); (b) influenza-like illness (RCTs: OR 0.51, 95% CI 0.19–1.41); or (c) reported workplace absenteeism (RCT: OR 0.92, 95% CI 0.57–1.50). In the surrogate exposure studies, N95 respirators were associated with less filter penetration, less face-seal leakage and less total inward leakage under laboratory experimental conditions, compared with surgical masks.
Interpretation: Although N95 respirators appeared to have a protective advantage over surgical masks in laboratory settings, our meta-analysis showed that there were insufficient data to determine definitively whether N95 respirators are superior to surgical masks in protecting health care workers against transmissible acute respiratory infections in clinical settings.
Transmission of acute respiratory infections occurs primarily by contact and droplet routes, and accordingly, the use of a surgical mask, eye protection, gown and gloves should be considered appropriate personal protective equipment when providing routine care for a patient with a transmissible acute respiratory infection.1–3 Concerns have been raised about possible acute respiratory infection spread via limited-distance airborne transmission, but this is controversial and has not been proven.1,4–9 Also, experimental data suggest the superiority of N95 filtering facepiece respirators (N95 respirators) over surgical masks for the prevention of acute respiratory infections.1 Randomized controlled trials (RCTs) and observational studies comparing N95 respirators and surgical masks have not shown a benefit, but they may have been underpowered.10–17
The lack of clarity has led to conflicting guideline recommendations regarding respiratory protective equipment for the prevention of acute respiratory infections: N95 respirators are recommended in some guidelines but not others.18 Since the outbreak of severe acute respiratory syndrome (SARS), there has been a heightened level of controversy within Canada in determining the optimal ways to protect health care workers from respiratory pathogens. Conflicting recommendations from federal and provincial health authorities lead to confusion among heath care workers, which can result in lack of adherence to basic infection control principles and practices.
We performed a systematic review to assess and synthesize the available body of literature regarding N95 respirators versus surgical masks for the protection of health care workers against acute respiratory infections in a health care setting.
Methods
A detailed protocol developed a priori is described in Appendix 1 (available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.150835/-/DC1).
Literature search
We searched MEDLINE, Embase, the Database of Abstracts of Reviews of Effects, the Cochrane Central Register of Controlled Trials, the Cochrane Database of Systematic Reviews, Health Technology Assessment, the Collective Index of Nursing and Allied Health Literature, PsycINFO and Scopus for pertinent English-language studies published from Jan. 1, 1990, to Dec. 9, 2014. (The search strategies are available in Appendix 1, Tables S1–S9.) The search start date marks 4 years before N95 respirators became a part of standard respiratory protective equipment among health care workers in the United States.
We also conducted searches of the grey literature to obtain unpublished data. These searches were limited to the past 5 years (see Appendix 1, Table S10, for search details).
Study selection
Randomized controlled trials, prospective and retrospective cohort studies, and case–control studies were eligible for inclusion in the meta-analysis. Participants in clinical studies were health care workers in a health care setting. We defined health care worker as any worker in a health care setting who might be exposed to a patient with an acute respiratory infection. We excluded studies that solely involved protection of patients or community populations.
Surrogate exposure studies (i.e., experiments involving manikins or volunteers exposed to artificially produced aerosols) were not eligible for inclusion in the meta-analysis but were summarized to provide an overview of the laboratory-based experimental evidence for use of N95 respirators to protect against acute respiratory infections. Aerosols are defined as a suspension of very small (0.01–100 μm in diameter) particles or droplets in the air.19 Studies with manikins or adult volunteers exposed to an aerosol simulating what might occur in a health care setting were considered.
Study designs assessed the use of National Institute for Occupational Safety and Health certified N95 respirators compared with surgical masks. Certification must have been under public health regulations (42 CFR part 84). Respirators certified under the former regulations (at 30 CFR part 11) were ineligible because they are no longer in use.20 We also included data on European standard filtering facepiece (FFP2) respirators (standards EN149:2001 and EN149:2001+A1:2009) as data on N95 filtering facepiece respirators. We did not include data on elastomeric facepiece respirators because they are not in widespread use in health care settings. The term “surgical mask” was considered equivalent to medical masks, procedural masks, isolation masks, laser masks, fluid-resistant masks and face masks that meet bacterial and particle filtration efficiency standards required by the US Food and Drug Administration (ASTM standard F2100–11) but are not certifiable as N95 respirators. Other types of respirators and surgical masks not explicitly described here were excluded.
Data extraction and quality assessment
The primary outcome of interest from RCTs, cohort studies and case–control studies was laboratory-confirmed respiratory infection, including respiratory infections diagnosed by means of polymerase chain reaction, serology, respiratory virus culture and Bordetella pertussis bacterial culture. Secondary outcomes were influenza-like illness, and workplace absenteeism due to hospital-acquired respiratory infections. The outcomes extracted from surrogate exposure studies were filter penetration, face-seal leakage and total inward leakage.
Two reviewers (J.D.S. and C.C.M.) independently screened abstracts, titles and full texts as described in the selection of studies. Data extraction was conducted using an electronic spreadsheet template (completed independently by J.D.S. and C.C.M.) and compared for discrepancies. Data from surrogate exposure studies were transformed, when appropriate, from fit-factors, protection factors or filter efficiencies to penetration percentages. When necessary, one of us (J.D.S.) contacted authors for additional information (Appendix 1, Table S11).
Randomized controlled trials were explicitly assessed for bias according to the Cochrane risk-of-bias tool.21 Cohort and case–control studies were assessed for risk of design-specific bias using the relevant Newcastle–Ottawa Scale.22
Outcome-specific quality of the body of evidence was assessed in duplicate by the same 2 reviewers using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework.23,24 Disagreements were resolved through consultation with a third reviewer (J.J.). The quality of evidence can be graded as high, moderate, low or very low.
Data synthesis
Where data could be combined for meta-analyses, these data were reported as odds ratios (ORs). We combined similar study designs only for the meta-analysis. Data were measured on dichotomous outcomes (laboratory-confirmed respiratory infection, influenza-like illness and workplace absenteeism). A random-effects analysis model and inverse variance statistical method were used for meta-analysis using Review Manager (RevMan).25
Cluster RCTs were adjusted for the meta-analysis with individual RCTs. We used the intraclass correlation coefficient to determine the design effect.26 Design effect was used to determine the effective sample size.26 When the effective sample size was not a whole number, it was rounded to the nearest whole number.
For meta-analyses involving rare events, zero cell counts were adjusted by including a correction (the reciprocal of the size of the contrasting study arm).27
We assessed evidence of heterogeneity using the χ2 test and I2 statistic; a χ2 value less than 0.10 or an I2 value greater than 50% indicated significant heterogeneity.28,29 Subgroup analysis was planned if there were more than 5 pooled studies and when significant heterogeneity was present.
All statistical analyses were performed with the use of RevMan (version 5.2; The Nordic Cochrane Centre, The Cochrane Collaboration, 2012).
Results
Search results and study characteristics
We screened 8962 titles, excluded 8855 and retrieved 107 articles for full-text assessment. We selected 31 eligible articles involving 29 studies; 6 were clinical studies that we included in the meta-analysis, and 23 were surrogate exposure studies (Figure 1). No unpublished abstracts of RCTs, cohort studies or case–control studies were found.
Selection of studies for the meta-analysis.
We included 3 RCTs, 1 cohort study and 2 case–control studies in the meta-analysis.11–17 The main characteristics of these studies are found in Table 1. All 6 studies reported laboratory-confirmed respiratory infection. Definitions of laboratory-confirmed respiratory infection differed. None of the RCTs used B. pertussis bacterial culture or viral culture. Neither of the RCTs by MacIntyre and colleagues12–14 used serology. The SARS cases in the cohort study15 and one of the case–control studies were confirmed only by serology.16 Zhang and colleagues17 confirmed influenza only by polymerase chain reaction. All of the RCTs reported on influenza-like illness. One RCT also reported workplace absenteeism; however, the outcome could not be confirmed to result from nosocomial respiratory infections.11
Effect on outcomes
No significant difference in risk of laboratory-confirmed respiratory infection was detected between health care workers using N95 respirators and those using surgical masks in the meta-analysis of the RCTs (OR 0.89, 95% confidence interval [CI] 0.64–1.24; I2 = 0%), the cohort study (OR 0.43, 95% CI 0.03–6.41) or the case–control studies (OR 0.91, 95% CI 0.25–3.36; I2 = 0%) (Figure 2). Similar results were found in 2 post-hoc meta-analyses: in one, we combined data from the 3 observational studies (OR 0.79, 95% CI 0.24–2.56; I2 = 0%); in the other, although not advised, we pooled data from all of the studies as an intellectual exercise to try to ascertain whether more precision could theoretically be obtained (OR 0.88, 95% CI 0.64–1.21; I2 = 0%).
Results of meta-analysis to determine effectiveness of N95 respirators versus surgical masks in protecting health care workers against acute respiratory infection. Outcomes were (A) laboratory-confirmed respiratory infection, (B) influenza-like illness and (C) workplace absenteeism. Values less than 1.0 favour N95 respirator. CI = confidence interval, NA = not applicable, RCT = randomized controlled trial.
We found no significant difference in risk of influenza-like illness between N95 respirators and surgical masks in the meta-analysis of the 3 RCTs (OR 0.51, 95% CI 0.19–1.41; I2 = 18%) (Figure 2). We also found no significant difference in risk of workplace absenteeism between N95 respirators and surgical masks in the 1 RCT that measured this outcome11 (OR 0.92, 95% CI 0.57–1.50) (Figure 2).
Risk of bias
The risk of bias for the RCTs is summarized in Figure S1 of Appendix 1. In brief, risk-of-bias ratings were identical across each domain of the Cochrane risk-of-bias tool for all included RCTs (low risk of bias for random sequence generation, incomplete outcome data, selective reporting and “other” bias; unclear risk of bias for allocation concealment; and high risk of bias for blinding of participants) except for blinding of outcome assessment, which was rated as unclear risk of bias for the RCT by Loeb and colleagues11 but as high risk of bias for the other 2 RCTs.12–14
Risk of bias for the cohort and case–control studies is summarized in Table S12 of Appendix 1. In brief, the cohort study15 received a rating of 6 stars, one of the case–control studies received 3 stars,16 and the other case–control study received 6 stars.17
Outcome-specific quality of evidence
The ratings of importance and outcome-specific quality of evidence that we assessed using the GRADE approach are summarized in Table S13 of Appendix 1. In brief, laboratory-confirmed respiratory infection was deemed a critically important outcome for decision-making with low-quality evidence from RCTs, and an important outcome for decision-making with very-low-quality evidence from observational studies. Influenza-like illness was rated as an important outcome for decision-making with very-low-quality evidence from RCTs. Work-related absenteeism was considered not an important outcome for decision-making with very-low-quality evidence from 1 RCT.
We did not conduct subgroup analyses because no significant heterogeneity was detected. No meaningful sensitivity analyses could be performed because too few studies were included.
Summary of surrogate exposure studies
Twenty-three surrogate exposure studies were included.30–53 Their outcomes and general methods (e.g., participants, particles used for exposure, number and type of respirator or surgical mask used, flow rates and breathing rates of manikins, size of challenge particles and range of particle size measured) are summarized in Appendix 2 (available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.150835/-/DC1). In general, compared with surgical masks, N95 respirators showed less filter penetration, less face-seal leakage and less total inward leakage under the laboratory experimental conditions described.
Interpretation
Results of our systematic review and meta-analysis show that there was no significant difference between N95 respirators and surgical masks when used by health care workers to prevent transmission of acute respiratory infections from patients. However, wide 95% CIs from our meta-analysis must be interpreted as insufficient evidence to determine whether there is a clinically significant difference. Findings from the surrogate exposure studies suggest that N95 respirators are superior to surgical masks for filter penetration, face-seal leakage and total inward leakage under laboratory conditions.
It was not surprising to find that N95 respirators were generally more efficient filters with better face-seal characteristics than surgical masks when tested in the laboratory. However, transmission of acute respiratory infections is a complex process that may not be appropriately replicated by surrogate exposure studies. Because the face seal is important for the efficiency of the N95 respirator, fit-testing is recommended for health care workers.2 N95 respirators are often considered uncomfortable for regular use, and improper wearing or adjustment of the respirator because of discomfort could lead to inadvertent face contamination, thus negating the potential protective benefit.54,55 Furthermore, we do not have an adequate understanding of the number, size and dispersion of the droplets that contain live, infectious particles produced by infected patients.56 A laboratory-based study reported data that humans infected with influenza rarely produce aerosols that contain infectious viral particles.57 In 2 other laboratory studies, participants infected with influenza produced droplets containing viral RNA, but viral RNA could not be detected on manikin headforms or on filters of breathing manikins at distances as close as 0.1 m following participants breathing, counting, coughing or laughing.7
Limitations
Despite our study’s many strengths, including a comprehensive search strategy for published data and grey literature, and a thorough review and assessment for risk of bias and quality of evidence using validated tools, limitations of this review should be acknowledged.
None of the studies included in the meta-analysis, except the RCT by Loeb and colleagues,11 independently audited compliance with the intervention. Potential confounding due to concurrent interventions (e.g., gloves, gowns and hand hygiene practices) as part of routine and additional precautions for droplet transmission were not accounted for by our meta-analysis.
We did not assess the impact of harms associated with mask and respirator use that could negatively affect the efficacy of the assigned intervention because it was out of the scope of our review.55
Acute respiratory infections may have been acquired during the study from community exposures rather than nosocomial exposure. In one of the RCTs,12,13 transmission may have occurred via contamination of provided respiratory protective equipment during storage and reuse of masks and respirators throughout the workday.
Only 2 respiratory virus seasons were assessed by the 3 RCTs; in one trial,14 the peak period of one of these influenza seasons was missed, and in another trial,11 the H1N1 outbreak in 2009 halted the study during the other respiratory season. Year-to-year strain variation of influenza necessitates additional data from other seasons during peak periods.
The weighting of the meta-analysis was influenced by the laboratory-confirmed respiratory infection outcome of serology used in one of the RCTs.11 However, health care workers who received influenza vaccination were appropriately excluded from analysis based only on serology.
Bias due to lack of blinding in all studies was a key factor in the relatively low GRADE quality assessment, and it is impossible to overcome because the health care workers would know which mask they were wearing.
Finally, these results are not generalizable to infections transmitted primarily through airborne routes (i.e., tuberculosis, measles and varicella) or to protection from acute respiratory infections during aerosol-generating medical procedures.3
Conclusion
Although N95 respirators appeared to have a protective advantage over surgical masks in laboratory settings, our meta-analysis showed that there were insufficient data to determine definitively whether N95 respirators are superior to surgical masks in protecting health care workers against transmissible acute respiratory infections in clinical settings. Additional, large RCTs are needed to detect a potentially clinically important difference owing to small event rates. Initial guidelines on preventing acute respiratory infection relied on surrogate exposure data and data extrapolated from the protection of health care workers against tuberculosis because clinical evidence did not exist at that time.58,59 Randomized controlled trials conducted in clinical settings represent the most valid information to evaluate the effectiveness of N95 respirators. They are more relevant to real clinical situations and report actual outcomes in health care workers, and therefore they are the best evidence on effectiveness to inform policy-making.
Acknowledgement
The authors are grateful for the contributions of the Library Services team at Public Health Ontario in the development, refinement and execution of the search strategy for this review.
Footnotes
-
Competing interests: None declared.
-
This article has been peer reviewed.
-
Contributors: Jeffrey Smith drafted the protocol and data extraction template, and performed the literature screening, data extraction and analysis. Colin MacDougall provided input for all stages of the project, and performed screening and data extraction. Jennie Johnstone provided input and made revisions on the protocol, data extraction template and analyses. Ray Copes, Brian Schwartz and Gary Garber were involved in the study conception. Gary Garber oversaw the project. Jeffrey Smith drafted the manuscript, and Colin MacDougall, Jennie Johnstone, Ray Copes, Brian Schwartz and Gary Garber provided critical feedback on the manuscript. All of the authors approved the final version to be published and agreed to act as guarantors of the work.
- Accepted December 2, 2015.