Automated Training Metrics System (ATMS)
A fundamental problem with cyber-security metrics is that the things we can easily quantify are rarely useful, and the things that are useful are hard to quantify. There have been significant efforts directed at providing a systematic methodology for assessing effectiveness of security policies and the performance of cyber security equipment and staff since these metrics is critical to the identification and mitigation of security deficiencies. They are also critical to the allocation of limited resources to provide the most efficient configuration possible.

Common Criteria is the most widely referenced assessment methodology for cyber security that, when properly applied, has demonstrated value in rating and ranking cyber security components and systems. However, the complexity of the Common Criteria methodology makes the process expensive and time consuming to apply. This high cost and lack of flexibility has consequently limited its application to very high value testing and evaluation problems. Also, since Common Criteria focuses on functional performance, it is often difficult to compare results from different studies, limiting its value in determining best practices.

The constant evolution of threats, tools, and tactics makes measurement of cyber security performance by staff one of the most challenging problems. Traditionally, trainers have focused on testing knowledge rather than skill to determine proficiency. High costs and short training cycles require the trainer to rapidly identify skill deficiencies so the student may revisit the topic, ultimately improving the overall training effectiveness. In general, current training metrics are subjective and specific to a given system of coursework, thereby limiting the ability to measure student capabilities across organizations.
 
Project Leads
Steve Ouderkirk, PNNL
Wayne Meitzler, PNNL

Return to Projects list


The National Center for Advanced Secure Systems Research NCASSR) at the University of Illinois at Urbana-Champaign (UIUC) is a multi-institutional cybersecurity research team. NCASSR focuses basic and applied research necessary to develop next generation information security technologies that address national cybersecurity needs. NCASSR is led by the National Center for Supercomputing Applications (NCSA) and supported by funding from the Office of Naval Research (ONR).


SELS 0.7 released
Secure Email List Services (SELS) is an open source software for creating and developing secure email list services among user communities.
 
Strong community engagement strengthens cybersecurity research and development
NCASSR-supported exploratory research at NCSA and elsewhere has sparked additional external funding and development opportunities as well as successful deployment and adoption by users ranging from the defense sector to state law enforcement to the utilities industry.
 
NCASSR Collaborator Goes To Washington
Carl Gunter, a professor in the University of Illinois Department of Computer Science and a project lead on NCASSR-supported work involving adaptive, secure messaging, recently spoke to an audience of congressional staffers and lobbyists on Capitol Hill regarding ways to address a variety of critical cybersecurity issues in areas such as healthcare and energy distribution.