What Does Lie-Detection Accuracy Measure? An Identification Audit of Behavioral LLM Deception Benchmarks
Accuracy on instructed-lie benchmarks is evidence about the instruction, not about deception detection.
What do LLM lie-detection benchmarks measure?
Two companion preprints. The first shows that accuracy on instructed-lie benchmarks is evidence about the instruction, not about deception. The second turns that argument into a reusable audit and tests it on published evaluations and on pre-registered experiments with label-free internal readouts.
What Do Behavioral Lie-Detection Benchmarks Identify? An Empirical Audit of Deception Detection
A reusable identification audit for lie-detection evaluations, applied to nine published evaluations and ten pre-registered experiments.
Code and data: github.com/anupamme/adaptive_lie_detector