Speaker
Description
Pipelines that turn security text into cyber threat intelligence (CTI) commonly translate non-English posts into English and then extract indicators with regular expressions and names with dictionaries. We show that the first step undermines the second. Machine translation silently alters file hashes, IP addresses and URLs while leaving them well-formed, so an extractor applied to translated text reports indicators that never existed. The effect appears in every open translation model we tested, from MarianMT to NLLB-200 and M2M100, is not fixed by larger models or beam search, and is only partly avoided by masking, whose success depends on the placeholder style the model happens to copy. Indicators therefore have to be taken from the source text. Using our own multilingual dark- web pipeline as the test case, we then measure the rest of the recipe against five CTI corpora with gold entity labels, under one protocol and on one GPU. Regular expressions are as good as anything we trained. Dictionaries of malware, tool and actor names miss most of what the corpora contain, and a small encoder fine-tuned for one or two minutes finds several times more entities than they do, while 8B language models sit in between at far higher cost. Models trained on one corpus lose much of their accuracy on another, because the corpora disagree about what a tool, a malware family or an indicator is. Keyword- based relevance filters look good against random Wikipedia text and fail against technical text about other topics, which a zero- shot entailment model handles without training. Along the way we restate our earlier headline figures on a defensible footing, including the overlap with the CISA KEV catalogue, which shrinks from a quarter to under three percent once the KEV feed itself is left out.