New detectors for nuclear, radiological material in cargo should not be acquired until testing deficiencies fixed, cost-benefit analysis completed (6/27/2009)

A thorough cost-benefit analysis that includes an assessment of meaningful alternatives is needed to reveal the potential security advantages of deploying new detector systems to screen cargo for nuclear and radiological materials at U.S. ports and border crossings. It is likely that the costs will exceed the savings gained from improved efficiency of the screening systems, says a new report from the National Research Council. There are shortcomings in the U.S. Department of Homeland Security's testing of these new detector systems, therefore the DHS secretary cannot conclude whether they will consistently outperform the current ones. DHS should not proceed with further procurement of these new detector systems until the issues are addressed and the systems have been shown to be a favored option in the cost-benefit analysis.

The 2006 SAFE Port Act requires that all containers coming into the United States through major entries be scanned for radiation, and "to the extent practicable, the Secretary shall deploy next generation radiation detection technology" to enable such scanning. In response, the department's Domestic Nuclear Detection Office (DNDO) requested proposals for advanced spectroscopic portals (ASPs), the next generation of radiation detectors for cargo screening, to replace the current system of radiation portal monitors and handheld radioisotope identifiers, which have known deficiencies. Before DHS can proceed with full-scale procurement of ASPs, Congress required the secretary to certify that they will provide a "significant increase in operational effectiveness" over continued use of the existing screening devices. If ASPs are certified, DHS may spend more than $1 billion to purchase these detector systems, with a possible net lifecycle cost of more than twice that figure. Currently, DHS is testing and evaluating the ASPs to inform the secretary's decision.

Congress asked the Research Council to advise DHS about testing, analysis, costs, and benefits of ASPs before this certification decision is made. The ASP testing and evaluation program encountered some delays in 2008, which created the opportunity for this interim report to provide advice on how testing, evaluation, and the cost-benefit analysis should be completed.

Testing of ASPs before 2008 had serious flaws, a number of which DNDO has acknowledged and addressed, said the committee that wrote the new interim report. In 2008, DNDO carried out physical tests to evaluate some of the limits of the ASP systems, but inadequacies remain. These include a lack of modeling to complement the physical tests and small test sample sizes that limit the confidence of comparison testing between the old and new detector systems.

The committee recommended an iterative testing approach, using computer models to simulate performance of the detector systems and physical experiments to test the models and identify needs for refinements. Then the needed model refinements could be undertaken. This iterative modeling and testing approach will allow DHS to gain a better understanding of the detector systems' performance, the committee said.

Rather than focusing on a single procurement to replace current screening technology, testing of ASPs should be viewed as a first step in improving and adapting the detector systems, the committee recommended. DHS should develop a process for continuous improvement that could address and exploit changes in technology and the nature of commerce so the system is not outdated or obsolete by the time it is fully deployed. DHS should begin this process by deploying the unused ASPs they already own in real ports of entry.

To determine whether the costs for these systems are reasonable and justified, a careful assessment will be needed to reveal the advantages of ASPs among alternatives, the committee said. The cost-benefit analysis should include a clear statement of the objectives of the program; an assessment of meaningful alternatives; and a comprehensive, credible, and transparent analysis of in-scope benefits and costs. The benefit assessment should show how the procurement contributes, relative to other possible DHS efforts and expenditures, to improving security with respect to prevention of the detonation of a nuclear or radiological device, which is the primary objective of the ASP program. A cost-benefit analysis that is silent on this subject would be incomplete, the committee noted.

The study was sponsored by U.S. Department of Homeland Security. The National Academy of Sciences, National Academy of Engineering, Institute of Medicine, and National Research Council make up the National Academies. They are independent, nonprofit institutions that provide science, technology, and health policy advice under an 1863 congressional charter. Committee members, who serve pro bono as volunteers, are chosen by the Academies for each study based on their expertise and experience and must satisfy the Academies' conflict-of-interest standards. The resulting consensus reports undergo external peer review before completion. For more information, visit http://national-academies.org/studycommitteprocess.pdf. A committee roster follows.

Note: This story has been adapted from a news release issued by the National Academy of Sciences

Post Comments: