Sandwiching evaluations test an oversight method on tasks where a model outperforms non-expert supervisors but experts can still check its answers. The experts supply the ground truth that would be unavailable if the model outperformed everyone supervising it (Cotra, 2021; Bowman et al., 2022).
Concept 94 of 100Scaling & AlignmentStudy phase 12
How to study this pagePredict · Check · Generalize · Continue
PredictSay what you expect.
Pick a concrete input and say what the equation should give before you read the math.
CheckCheck against the math and the papers.
Work through the math, then read the cited papers for the original argument.
GeneralizeName what stays the same.
Identify the part of the idea that survives a change of notation or model, so you can recognize it in another paper or system.
ContinueFollow the connections.
Use the prerequisites and related concepts at the end of the page to fill the next gap.
Why It Matters for Modern Models
Sandwiching evaluations test an oversight method on tasks where a model outperforms non-expert supervisors but experts can still check its answers. The experts supply the ground truth that would be unavailable if the model outperformed everyone supervising it (Cotra, 2021; Bowman et al., 2022).
In a first experiment on MMLU and time-limited QuALITY questions, people who chatted with an unreliable language-model assistant substantially outperformed both the model alone and their own unaided performance (Bowman et al., 2022).
The design only works for tasks where experts can grade reliably while non-experts struggle, and where the model really is more capable than the non-experts (Bowman et al., 2022).
Questions to Think About
If assisted non-experts do well, how can you tell whether the oversight method worked or the people simply copied the model's answers?
What would a sandwiching experiment need in order to test oversight of a model that is trying to mislead its overseers?
Core Math
Key Equation
max(PH,PM)<PH+M≤PE
Sandwiching tests an oversight method on a task where the model sits between two groups of people: it is more capable than the non-experts who will supervise it, but less capable than experts who can check the results (Cotra, 2021; Bowman et al., 2022).
PH: accuracy of non-experts working alone
PM: accuracy of the model alone
PH+M: accuracy of non-experts working with the model, using the oversight method under test
PE: accuracy of experts, who are used only to grade
The method succeeds if assisted non-experts beat both the model and themselves alone, and get close to the experts:
max(PH,PM)<PH+M≤PE
One way to summarize a result, an illustrative normalization rather than one defined in these papers, is the fraction of the gap to the experts that the assistance closes, (PH+M−PH)/(PE−PH): 0 when assistance does not help, and 1 when assisted non-experts match the experts.
Key Papers
Measuring Progress on Scalable Oversight for Large Language Models