Speaker
Description
Large language models (LLMs) are increasingly used to map Cyber Threat Intelligence (CTI) text to MITRE ATT&CK techniques, but they can also return techniques that are not supported by the source text. We investigate whether prompt design can reduce these unsupported predictions. Qwen3-32B maps the same 228 annotated CTI paragraphs under three prompting strategies: direct mapping, evidence-constrained mapping that requires a verbatim supporting span, and behavior-first mapping that describes the adversary behavior before predicting the technique. The prompts are finalized on a validation subset before evaluation on the held-out test split. We find no statistically significant difference in gold-mapping recovery between the three strategies (exact McNemar test, p > 0.5 for all pairwise comparisons), while the unsupported prediction rate remains between 0.89 and 0.91. The evidence constraint is followed correctly: 43 of the 44 correct mappings under this condition cite text that appears verbatim in the paragraph, with a mean token Jaccard of 0.55 against the annotated evidence span. However, this additional grounding does not improve technique-level accuracy. Recall is instead strongly associated with how the technique is expressed in the text: lexically signaled techniques are recovered 2.3 to 3.4 times more often than behavioral mappings across all three prompts. Requiring evidence improves the verifiability of ATT&CK mappings, but does not by itself improve their correctness.