JuStRank: Benchmarking LLM Judges for System Ranking
Ariel Gera, Odellia Boni, et al.
ACL 2025
A central question in understanding human language is how people store, access, and comprehend words. The ongoing COVID-19 pandemic presented a natural experiment to investigate whether language comprehension can be changed in a lasting way by external experiences. We leveraged the sudden increase in the frequency of certain words (mask, isolation, lockdown) to investigate the effects of rapid contextual changes on word comprehension, measured over 10 months within the first year of the pandemic. Using the phonemic restoration paradigm, in which listeners are presented with ambiguous auditory input and report which word they hear, we conducted four online experiments with adult participants across the United States (combined N = 899). We find that the pandemic has reshaped language processing for the long term, changing how listeners process speech and what they expect from ambiguous input. These results show that abrupt changes in linguistic exposure can cause enduring changes to the language system.
Ariel Gera, Odellia Boni, et al.
ACL 2025
Keerthiram Murugesan, Mattia Atzeni, et al.
AAAI 2021
Mateus Do Amor Devino Pereira, Evaline Ju, et al.
CAIN 2025
Amar Prakash Azad, Supriyo Ghosh, et al.
IAAI 2022