Skip to main content
archive
Search Submit Donate Log in

Computer Science > Computation and Language

(cs)
[Submitted on 7 Sep 2023 (v1), last revised 22 Feb 2024 (this version, v4)]

Title:Large Language Models Are Not Robust Multiple Choice Selectors

Authors:Chujie Zheng, Hao Zhou, Fandong Meng, Jie Zhou, Minlie Huang
View a PDF of the paper titled Large Language Models Are Not Robust Multiple Choice Selectors, by Chujie Zheng and 4 other authors
View PDF HTML (experimental)
Abstract:Multiple choice questions (MCQs) serve as a common yet important task format in the evaluation of large language models (LLMs). This work shows that modern LLMs are vulnerable to option position changes in MCQs due to their inherent "selection bias", namely, they prefer to select specific option IDs as answers (like "Option A"). Through extensive empirical analyses with 20 LLMs on three benchmarks, we pinpoint that this behavioral bias primarily stems from LLMs' token bias, where the model a priori assigns more probabilistic mass to specific option ID tokens (e.g., A/B/C/D) when predicting answers from the option IDs. To mitigate selection bias, we propose a label-free, inference-time debiasing method, called PriDe, which separates the model's prior bias for option IDs from the overall prediction distribution. PriDe first estimates the prior by permutating option contents on a small number of test samples, and then applies the estimated prior to debias the remaining samples. We demonstrate that it achieves interpretable and transferable debiasing with high computational efficiency. We hope this work can draw broader research attention to the bias and robustness of modern LLMs.
Comments: ICLR 2024 Spotlight
Subjects: Computation and Language (cs.CL)
Cite as: arXiv:2309.03882 [cs.CL]
  (or arXiv:2309.03882v4 [cs.CL] for this version)
  https://doi.org/10.48550/arXiv.2309.03882
arXiv-issued DOI via DataCite

Submission history

From: Chujie Zheng [view email]
[v1] Thu, 7 Sep 2023 17:44:56 UTC (2,433 KB)
[v2] Fri, 8 Sep 2023 15:54:56 UTC (2,433 KB)
[v3] Fri, 6 Oct 2023 08:27:26 UTC (3,736 KB)
[v4] Thu, 22 Feb 2024 01:40:35 UTC (3,738 KB)
Full-text links:

Access Paper:

    View a PDF of the paper titled Large Language Models Are Not Robust Multiple Choice Selectors, by Chujie Zheng and 4 other authors
  • View PDF
  • HTML (experimental)
  • TeX Source
view license

Current browse context:

cs.CL
< prev   |   next >
new | recent | 2023-09
Change to browse by:
cs

References & Citations

  • NASA ADS
  • Google Scholar
  • Semantic Scholar
Loading...

Bookmark

BibSonomy Reddit

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)
We gratefully acknowledge support from our major funders, member institutions, , and all contributors.
About · Help · Contact · Subscribe · Copyright · Privacy · Accessibility · Operational Status (opens in new tab)
Major funding support from
Simons Foundation Simons Foundation International Schmidt Sciences