Elicitation by Critiquing: Applications to Computer Network Defense

With an increasing frequency of data breaches suffered by organizations, computer network defense (CND) is becoming an increasingly important concern. With understanding of how cybersecurity professionals engage in the cognitive aspects of their work, human factors researchers and practitioners can improve tools and training. By optimizing the tools and training network defenders rely on to detect and respond to novel network threats, the cybersecurity workforce will be strengthened. While cognitive task analysis (CTA) is well-positioned to represent the cognitive work of CND, we identify challenges practitioners are likely to encounter in the field. Through a review of published CTAs in CND and other domains, we provide guidance for future CTA efforts in CND. Finally, we present an argument for the use of Elicitation by Critiquing (EBC) and demonstrate its efficacy in mitigating the challenges of applying CTA in CND.

Ecological Interface Design for Computer Network Defense

Objective: A prototype ecological interface for computer network defense (CND) was developed. Background: Concerns about CND run high. Although there is a vast literature on CND, there is some indication that this research is not being translated into operational contexts. Part of the reason may be that CND has historically been treated as a strictly technical problem, rather than as a socio-technical problem. Methods: The cognitive systems engineering (CSE)/ecological interface design (EID) framework was used in the analysis and design of the prototype interface. A brief overview of CSE/EID is provided. EID principles of design (i.e., direct perception, direct manipulation and visual momentum) are described and illustrated through concrete examples from the ecological interface. Results: Key features of the ecological interface include (a) a wide variety of alternative visual displays, (b) controls that allow easy, dynamic reconfiguration of these displays, (c) visual highlighting of functionally related information across displays, (d) control mechanisms to selectively filter massive data sets, and (e) the capability for easy expansion. Cyber attacks from a well-known data set are illustrated through screen shots. Conclusion: CND support needs to be developed with a triadic focus (i.e., humans interacting with technology to accomplish work) if it is to be effective. Iterative design and formal evaluation is also required. The discipline of human factors has a long tradition of success on both counts; it is time that HF became fully involved in CND. Application: Direct application in supporting cyber analysts.

Task Interruptions Undermine Cyber Defense

Computer network defense analysts engage a difficult, though critical, task in cyber defense. Anecdotally, these operators complain of frequent task interruptions while they are performing their duties. The goal for the current study was to investigate the effect of a commonly reported interruption, answering email, on accuracy and completion times in a simulated network analyst task. During task trials, participants were interrupted by emails between alert investigations, during alert investigations, or not at all (control). The results indicated that email interruptions increased alert completion times regardless of when they occurred, but interruptions that occurred during an alert investigation also reduced the accuracy of subsequent judgments about alert threat. Overall, the results suggest that task interruptions can potentially undermine cyber defense, and steps should be taken to better quantify and mitigate this threat.