In 2005 Secure Decisions was contracted by the Intelligence Community's Intelligence Advanced Research Projects Activity (IARPA, formerly the Advanced Research and Development Activity, or ARDA), under the Proactive and Predictive Information Assurance for Next Generation Systems (P2INGS) program, to study and report on the work processes of Computer Network Defense (CND) analysts. We organized and conducted a two-phase Cognitive Task Analysis (CTA) to achieve this goal. The published results of this CTA have been widely requested across the government community, and have proven to be a valuable resource to support research and development of new CND tools and practices.

CND Cognitive Task Analysis (CTA), Phase 1 [1] – Investigators interviewed and observed network defenders responsible for various aspects of cyber defense in six organizations within the US Department of Defense (DoD), and one commercial Managed Service Security Provider (MSSP). The results of this work are being applied to the design of visual data presentation methods that can enhance analysts’ ability to process large volumes of network data, rapidly detect suspicious activities, correlate multiple data sources, and forecast future threats.

CND Cognitive Task Analysis (CTA), Phase 2 – A follow-on effort under the same sponsorship, this phase of the CTA was intended to:

• Determine how CND analysts are trained and provide recommenda­tions to improve the training process.

• Characterize the skill level attributes of CND analysts along a spectrum from beginner to expert; identify how CND effectiveness is measured and distill a generally applicable set of metrics.

• Identify organizational and procedural trends that enhance or retard the collective effectiveness of CND operations.

The work done under this program has provided Secure Decisions with unique insight into the work-centered knowledge acquisition and skills development problems faced by CND practitioners. The observations from both CTA projects point to very specific challenges that relate to the effectiveness of CND decision-making, with direct implications for CND training and simulation. This insight has driven the development of all of our CND tools and techniques.