DHS asks for help designing the cyber testbed of the future

A graphic representation of the Cyber Experimentation for the Future, or CEF, Testbed (Source: NSF)

The Department of Homeland Security is asking for suggestions from vendors about a new cybersecurity laboratory it wants to set up called the Cyber Experimentation for the Future, or CEF, Testbed.

In a Request for Information published Thursday, the DHS Science and Technology Directorate’s Cyber Security Division says it wants input “from industry, academia and other interested stakeholders” about what would be involved in “operating and maintaining a ‘CEF testbed'” and what organizational structure might best enable “the development of new [research and development] tools by industry, academia, national labs, international researchers, etc., for the CEF testbed.”

Among the information DHS officials say they are looking for:

• “Capabilities to establish and operate a large CEF testbed with hundreds to thousands of nodes;”
• “Facilities to maintain availability and continuity of operations of hardware and software twenty-four hours per day and seven days per week;”
• Ways of increasing the “portability” of experiments;
• A “flexible and dynamic test automation framework;”
• Models and tools which work in both real and simulated environments, and that “reflect real world situations in order to increase the probability that experimental results will solve real world cybersecurity problems;”
• Approaches that support “cross domain experimentation” in cybersecurity across various sectors of vital industry; and
• Outreach to the research community to let them know about “the testbed and its capabilities, and to increase the utilization of the infrastructure.’

The RFI says it is based on a National Science Foundation-funded report published last year called “Cybersecurity Experimentation of the Future (CEF): Catalyzing a New Generation of Experimental Cybersecurity Research.”

According to the report’s website, it “explored current and expected experimentation infrastructure needs” for cybersecurity research into the future, and it “produced a roadmap for developing an accessible, broad, and multi-organizational cybersecurity experimentation capability that supports tomorrow’s research.”

The report came up with five top recommendations:

• Multidisciplinary experimentation;
• Inclusion of human activity for real world, scientifically sound experiments;
• Ensuring that whatever’s built has open interfaces “to support extensibility;”
• Reusable experiment designs for science-based hypothesis testing; and
• Infrastructure usability and research community cultural changes.

DHS already funds a cyber testbed called the Cyber Defense Technology Experimental Research Laboratory, or DETERLab, that over 220 organizations from 30 different countries including local and national governments, businesses and dozens of academic institutions have used it to analyze cybersecurity solutions.”

DETERLab is based at the University of Southern California. “Our adversaries have an incredible environment for testing out attacks: the internet, on which all our production systems operate,” said Terry Benzel, deputy director for the Internet and Networked Systems Division at the USC Information Sciences Institute.

“They can sit and analyze our vulnerabilities for as long as they want, probe and poke and run experiments until they find the right way in,” she said in a National Science Foundation release last year.

Benzel was one of the lead authors on the NSF report.

Clifford Neuman, director of USC Center for Computer System Security, adds, in this YouTube video, “DETER allows us to model the internet so that we can understand how our systems will respond in the real world.”

The DETER system allows researchers to “replay scenarios and attacks, looking at what configurations and solutions work the best against attackers,” according to Luke Berndt, a former Department of Homeland Security system manager.

DHS officials says CEF will be the next generation testbed, beyond DETER.

“There are things that we don’t do with DETER that we want in CEF,” said one official, listing industrial control system, or ICS, modeling, “new domains of applicability (e.g., IOT), modeling human behavior in cyberspace, more open and standard interfaces, and new tools for experimentation management.”

USC officials declined comment on the future of DETER.