Report in Brief - Division on Earth and Life Studies
Transcription
Report in Brief - Division on Earth and Life Studies
BOARD ON LIFE SCIENCES, DIVISION ON EARTH AND LIFE STUDIES BOARD ON HEALTH SCIENCES POLICY, INSTITUTE OF MEDICINE MAY 2015 BioWatch PCR Assays: Building Confidence, Ensuring Reliability BIOWATCH IS AN AIR MONITORING SYSTEM deployed in jurisdictions around the country with the goal of detecting the presence of certain high risk pathogenic microorganisms. It relies on a network of federal and nonfederal collaborative relationships to be successful, and is one part of a larger array of disease surveillance, intelligence- gathering, and biomonitoring activities in support of public safety and health. The assays used in the BioWatch system to detect the presence of pathogens in collected samples rely on the technique of polymerase chain reaction (PCR) to sensitively and specifically amplify target nucleic acid sequences. The program and its users need an understanding of each assay’s performance characteristics in order to have confidence in the results and have the ability to appropriately interpret them. This confidence is partly provided through a performance standard, which gives information on the minimum requirements that must be met for the assay to be considered acceptable for its intended purpose and describes how testing to validate this performance is to be carried out. The National Research Council convened a committee of experts to examine standards for PCR assays appropriate for the BioWatch program. The committee’s report discusses principles of performance standards, reviews information from several existing guidance documents and standards that might be applicable to BioWatch, and discusses assay testing efforts that have occurred or are ongoing. It provides recommendations on general principles and approaches for a performance standard and validation framework to meet BioWatch’s mission. It also provides a starting point for an approach to in silico and laboratory assay characterization for consideration by BioWatch and its stakeholders, with the aim of providing a reasonable amount of information on performance parameters such as an assay’s limit of detection, sensitivity, and specificity. As tasked, the report focuses on the PCR reaction portion of a PCR assay; although it discusses key processes before and after the PCR, these are not addressed in detail. One fundamental aspect of assay validation for a system such as BioWatch should be recognized and clearly communicated to program stakeholders—BioWatch is designed to detect rare but high-consequence events and all assays will have some associated false positive and false negative rates. It is not practical in time, effort, or money to conduct laboratory-based assay performance testing using large enough numbers of replicates under enough different conditions to validate the assay’s long-term false positive or false negative rate at the high levels of statistical confidence program users are likely to desire for operational decision making (e.g., to determine with statistical confidence that a false result will not occur more than once per x thousands or hundreds of thousands of samples). The assay design and laboratory performance validation that is the focus of the present report thus provides initial information that the assay is anticipated to perform reasonably well. It is critical for the system’s users to understand the testing that has been undertaken and the limitations of the data, and to combine this knowledge with ongoing analysis of data obtained from verification in jurisdictional laboratories and from operational deployment of the assay in order to achieve confidence in the results and to facilitate responses to positive BioWatch detections. The report emphasizes the importance of communication and, particularly, of discussing the details of performance data with jurisdictional laboratory experts and officials. The report also emphasizes the importance of obtaining a better understanding of the microorganism background that is present in BioWatch jurisdictions—information that remains very limited. Finally, the report considers how developments in technology, particularly in multiplex PCR and nextgeneration sequencing (NGS), can contribute to the ability of the BioWatch program to meet current and future challenges. Sequencing currently can be used to follow up on unexpected assay results from the jurisdictions. In the nearer term, targeted approaches coupled with NGS may be useful as a replacement for the current real-time PCR secondary assays because of their ability to analyze many more genomic regions for identification and characterization. The applicability of techniques such as metagenomic NGS to the program would be a longer-term prospect. The report recommends that the Department of Homeland Security and the BioWatch program continue to monitor and evaluate technologies as they develop. The program also should plan to work with laboratory users in the BioWatch jurisdictions, along with technology experts, to ensure that new technology brought into the program not only incorporates the best technology but also functions smoothly for the stakeholder community. Locate additional information, including related reports, at http://dels.nas.edu/bls Read, purchase, or download a free PDF of this report at http://www.nap.edu Committee on PCR Standards for the BioWatch Program: Georges C. Benjamin (Co-Chair), Executive Director, American Public Health Association, Washington, DC; Kenneth I. Berns (Co-Chair), Distinguished Professor of Molecular Genetics and Microbiology Emeritus, University of Florida College of Medicine, Gainesville; Bruce Budowle, Executive Director, Institute of Applied Genetics, and Professor, Department of Molecular and Medical Genetics, University of North Texas Health Science Center, Fort Worth; Charles Chiu, Associate Professor, Laboratory Medicine and Medicine, Infectious Diseases; Director, UCSF-Abbott Viral Diagnostics and Discovery Center at China Basin; and Associate Director, UCSF Clinical Microbiology Laboratory, University of California, San Francisco; John M. Hardham, Associate Research Fellow and Chair, Emerging Infectious Disease Program, Zoetis, Inc., Kalamazoo, MI; Grace Kubin, Director, Laboratory Services Section, Texas Department of State Health Services, Austin; M. Allen Northrup, Principal, Northrup Consulting Group, San Francisco, CA; Tom Slezak, Program Leader, Global Security Program, E.O. Lawrence Livermore National Laboratory, Livermore, CA; Peter M. Vallone, Leader, Applied Genetics Group, Biomolecular Measurement Division, National Institute of Standards and Technology, Gaithersburg, MD; Katherine Bowman (Study Director, Board on Life Sciences), Autumn Downey (Program Officer, Board on Health Sciences Policy), Kathryn Hughes (Senior Program Officer, Board on Chemical Sciences and Technology), Jon Q. Sanders (Program Coordinator, Board on the Health of Select Populations–until November 2014), Nia D. Johnson (Senior Research Associate, Board on Army Science and Technology–from November 2014), Bethelhem Mekasha (Financial Associate, Board on Life Sciences), Andrew Pope (Director, Board on Health Sciences Policy), Frances Sharples (Director, Board on Life Sciences), National Research Council The National Academies appointed the above committee of experts to address the specific task requested by the Department of Homeland Security. The members volunteered their time for this activity; their report is peer-reviewed and the final product signed off by both the committee members and the National Academies. This report brief was prepared by the National Research Council based on the committee’s report. For more information, contact the Board on Life Sciences at (202) 334-2187 or visit http://dels.nas.edu/bls. Copies of BioWatch PCR Assays: Building Confidence, Ensuring Reliability are available from the National Academies Press, 500 Fifth Street, NW, Washington, D.C. 20001; (800) 624-6242; www.nap.edu. Permission granted to reproduce this document in its entirety with no additions or alterations. Permission for images/figures must be obtained from their original source. © 2015 The National Academy of Sciences