19–21 Nov 2026
"Alexandru Ioan Cuza" University of Iasi and "Gheorghe Asachi" Technical University of Iași
Europe/Bucharest timezone

Does Prompt Design Affect MITRE ATT&CK Mapping from CTI?

20 Nov 2026, 09:30
15m
Room 2

Room 2

Paper presentation Doctoral Symposium Doctoral Symposium

Speaker

Ebru Resul

Description

Large language models (LLMs) are increasingly used to map Cyber Threat Intelligence (CTI) text to MITRE ATT&CK techniques, but they can also return techniques that are not supported by the source text. We investigate whether prompt design can reduce these unsupported predictions. Qwen3-32B maps the same 228 annotated CTI paragraphs under three prompting strategies: direct mapping, evidence-constrained mapping that requires a verbatim supporting span, and behavior-first mapping that describes the adversary behavior before predicting the technique. The prompts are finalized on a validation subset before evaluation on the held-out test split. We find no statistically significant difference in gold-mapping recovery between the three strategies (exact McNemar test, p > 0.5 for all pairwise comparisons), while the unsupported prediction rate remains between 0.89 and 0.91. The evidence constraint is followed correctly: 43 of the 44 correct mappings under this condition cite text that appears verbatim in the paragraph, with a mean token Jaccard of 0.55 against the annotated evidence span. However, this additional grounding does not improve technique-level accuracy. Recall is instead strongly associated with how the technique is expressed in the text: lexically signaled techniques are recovered 2.3 to 3.4 times more often than behavioral mappings across all three prompts. Requiring evidence improves the verifiability of ATT&CK mappings, but does not by itself improve their correctness.

Author

Ebru Resul

Co-authors

Caius Rughiniș (Politehnica Bucharest, Romania) Mihai Țurcanu (Faculty of Economic Engineering and Business Technical University of Moldova) Mr Văduva Alexandru Jan (Politehnica Bucharest, Romania)

Presentation materials

There are no materials yet.