The good, the bad and the ugly: a study of security decisions in a cyber-physical systems game

Abstract

Stakeholders' security decisions play a fundamental role in determining security requirements, yet, little is currently understood about how different stakeholder groups within an organisation approach security and the drivers and tacit biases underpinning their decisions. We studied and contrasted the security decisions of three demographics -- security experts, computer scientists and managers -- when playing a tabletop game that we designed and developed. The game tasks players with managing the security of a cyber-physical environment while facing various threats. Analysis of 12 groups of players (4 groups in each of our demographics) reveals strategies that repeat in particular demographics, e.g., managers and security experts generally favoring technological solutions over personnel training, which computer scientists preferred. Surprisingly, security experts were not ipso facto better players -- in some cases, they made very questionable decisions -- yet they showed a higher level of confidence in themselves. We classified players' decision-making processes, i.e., procedure-, experience-, scenario- or intuition-driven. We identified decision patterns, both good practices and typical errors and pitfalls. Our game provides a requirements sandbox in which players can experiment with security risks, learn about decision-making and its consequences, and reflect on their own perception of security.