Abstract
This paper proposes a novel approach for the evaluation of the performance achieved by trainees involved in cyber security exercises implemented in modern cyber ranges. Our main contributions include: the definition of a distributed monitoring architecture for gathering relevant information about trainees activities; an algorithm for modeling the trainee activities using directed graphs; novel scoring algorithms, based on graph operations, that evaluate different aspects (speed, precision) of a trainee during an exercise. With respect to previous work, our proposal allows to measure exactly how fast a user is progressing towards an objective and where he does wrong. We highlight that this is currently not possible in the most popular cyber ranges.














Similar content being viewed by others
References
Bagnato A, Kordy B, Meland PH, Schweitzer P (2012) Attribute decoration of attack–defense trees. Int J Secur Softw Eng (IJSSE) 3(2):1–35
Bowen BM, Devarajan R, Stolfo S (2011) Measuring the human factor of cyber security. In: 2011 IEEE International conference on technologies for homeland security (HST). IEEE, pp 230–235
Carlisle M, Chiaramonte M, Caswell D (2015) Using ctfs for an undergraduate cyber education. In: 2015 {USENIX} summit on gaming, games, and gamification in security education (3GSE 15)
Čeleda P, Čegan J, Vykopal J, Tovarňák D (2015) Kypo–a platform for cyber defence exercises. M&S Support to Operational Tasks Including War Gaming, Logistics,Cyber Defence. NATO Science and Technology Organization
CISCO Cyber Range (2016) https://www.cisco.com/c/dam/global/en_au/\solutions/security/pdfs/cyber_range_aag_v2.pdf
Evans M, He Y, Maglaras L, Janicke H (2019) Heart-is: a novel technique for evaluating human error-related information security incidents. Comput Secur 80:74–89
Ferguson B, Tall A, Olsen D (2014) National cyber range overview. In: 2014 IEEE Military communications conference. IEEE, pp 123–128
Huang K, Siegel M, Stuart M (2018) Systematically understanding the cyber attack business: a survey. ACM Comput Surv (CSUR) 51(4):70
IXIA Cyber Range (2014) https://www.ixiacom.com/solutions/cyber-range
Jameel A, Shahzad K, Zafar A, Ahmed U, Hussain SJ, Sajid A (2018) The users experience quality of responsive web design on multiple devices. In: Proceedings of the 2nd international conference on future networks and distributed systems. ACM , p 69
Kordy B, Kordy P, Mauw S, Schweitzer P (2013) Adtool: security analysis with attack–defense trees. In: International conference on quantitative evaluation of systems. Springer, pp 173–176
Kraemer S, Carayon P, Clem J (2009) Human and organizational factors in computer and information security: pathways to vulnerabilities. Comput Secur 28(7):509–520
Lampesberger H (2016) Technologies for web and cloud service interaction: a survey. SOCA 10(2):71–110
Mauw S, Oostdijk M (2005) Foundations of attack trees. In: International conference on information security and cryptology. Springer, pp 186–198
Miehling E, Rasouli M, Teneketzis D (2015) Optimal defense policies for partially observable spreading processes on bayesian attack graphs. In: Proceedings of the second ACM workshop on moving target defense. ACM, pp 67–76
Ou X, Boyer WF, McQueen MA (2006) A scalable approach to attack graph generation. In: Proceedings of the 13th ACM conference on computer and communications security. ACM, pp 336–345
Pernik P (2014) Improving cyber security: Nato and the eu International Centre for Defense Studies
Poolsappasit N, Dewri R, Ray I (2011) Dynamic security risk management using Bayesian attack graphs. IEEE Trans Depend Secur Comput 9(1):61–74
Schneier B (1999) Attack trees. Dr Dobb’s J 24(12):21–29
Sheyner O, Haines J, Jha S, Lippmann R, Wing JM (2002) Automated generation and analysis of attack graphs. In: Proceedings 2002 IEEE symposium on security and privacy. IEEE , pp 273–284
Vykopal J, Vizváry M, Oslejsek R, Celeda P, Tovarnak D (2017) Lessons learned from complex hands-on defence exercises in a cyber range. In: 2017 IEEE frontiers in education conference (FIE). IEEE, pp 1–8
Zonouz SA, Khurana H, Sanders WH, Yardley TM (2013) Rre: a game-theoretic intrusion response and recovery engine. IEEE Trans Parallel Distrib Syst 25(2):395–406
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Andreolini, M., Colacino, V.G., Colajanni, M. et al. A Framework for the Evaluation of Trainee Performance in Cyber Range Exercises. Mobile Netw Appl 25, 236–247 (2020). https://doi.org/10.1007/s11036-019-01442-0
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11036-019-01442-0