Speaker
Description
In this paper, we present a system for fake news detection in Romanian, a low-resource language with few manually annotated news articles and where real news prevails among available data. The task is classification of news articles into four classes - real news and three types of misinformation - through three iterations. The first iteration fine-tunes Romanian BERT encoder models with convolutional classification heads on about 760 COVID-19 articles from the FakeRom corpus, manually labeled by domain experts, resulting in 84.2% accuracy and weighted F1 of 0.83. In the second iteration, we go to a much larger corpus and a new four-class taxonomy, use generative large language models (LLMs), RoMistral-7b and RoLlama2-7b, and fine-tune these models with Low-Rank Adaptation (LoRA). We evaluate how 4-bit quantization and prompt phrasing impact our results. RoMistral-7b gets to the weighted F1 of 0.85 and quantization incurs penalty of six to seven points in both models. The third iteration returns us to an encoder with a fully fine-tuned XLM-RoBERTa-large classifier, which we evaluate using ten configurations with only one variable changed each time - the backbone model, loss functions for handling the class imbalance, two types of data augmentations, external data from SaRoCo satire corpus, and whether class descriptions are included in the input. The best configuration reaches a weighted F1 of 0.88 while training in under 15 minutes, compared to two to three hours for the generative models on the same hardware. Another comparison is made between the above configuration and the best commercially available
LLMs such as GPT, Gemini, and Claude, which are prompted instead of fine-tuned on the four-class task. These models have not been exposed to the corpus or adapted for Romanian misinformation categories, and this comparison seeks to determine if the general-purpose scale holds its own.