19–21 Nov 2026
"Alexandru Ioan Cuza" University of Iasi and "Gheorghe Asachi" Technical University of Iași
Europe/Bucharest timezone

An Auditable, Evidence-Based Multi-Level Framework for Automated Assessment of Cybersecurity Hands-On Labs

20 Nov 2026, 10:00
15m
Room 1

Room 1

Paper presentation Networking in Education and Research Networking in Education and Research

Speaker

Arina Alexei (Technical University of Moldova)

Description

Manual evaluation of terminal logs in cybersecurity labs is difficult for large cohorts, especially in the absence of a centralized infrastructure. Checking the presence of a command in the terminal history cannot confirm that the task was completed. We propose an auditable evaluation framework, based on 4 types of evidence extracted from logs: command, argument, verification and output. The evidence was combined cumulatively on 3 levels of confirmation: syntactic (L1), semantic (L2) and confirmed technical result (L3), corresponding to the log fragment. The framework was tested on 899 eligible logs obtained from 6 heterogeneous labs: network scanning, access control, cryptography, digital signatures, VPN and firewall. For the direct comparison, 550 logs containing the full session transcript were used, with a total of 7192 decisions made, with average scores of 85.34% (L1), 58.59% (L2), and 39.68% (L3). A difference of 45.66 percentage points and a very large effect size (Kendall's W=0.880) was recorded. Procedural tasks had higher scores than multi-stage ones. The framework is sensitive to partial log loss or truncation and invariant to ANSI formatting tests, after data normalization, processes 8.71 logs/second and supports automatic initial cohort assessment, in the absence of centralized collection infrastructure.

Authors

Arina Alexei (Technical University of Moldova) Anatolie Alexei (Technical University of Moldova) Mr Daniel-Ilie SÎRBU (UPB) Bogdan-Costel Mocanu (UPB)

Presentation materials

There are no materials yet.