Speaker
Description
Auditing algorithmic amplification on very largeonline platforms, as envisioned under Article 40 of the EU DigitalServices Act, is obstructed by two constraints: auditors rarelyhave white-box access to production rankers, and counterfactualinterventions cannot be run on a live platform. A naturalworkaround is to distill a surrogate of the ranker and studyit inside an agent-based simulation, but this is only soundif conclusions transfer from surrogate to target. We test thistransfer in OASIS, an LLM-agent simulation of X/Twitter whoseranker is TwHIN-BERT. We distill two surrogates under differentaccess assumptions, one reusing the target’s first four transformerlayers and one training a compact sentence encoder on thetarget’s embeddings, and run matched simulations in whichonly the ranker differs. When a single post spreads throughthe population of its real cascade, all rankers reproduce thescale of Twitter15/16 cascades equally well. When 100 postscompete for two recommendation slots, the surrogates share only9–29% of their first recommendations with their teacher, andoutcomes such as the share of reposts on false rumors varyacross rankers. A minor implementation change in the targetitself, its position-embedding handling, reduces agreement withthe deployed ranker to 7%. Embedding-level fidelity thereforedoes not guarantee fidelity of exposure outcomes, and single-postcascades are a weak test of a ranker. These preliminary, single-seed results motivate validating surrogate-based audits in settingswhere the ranker actually filters content.