When Agreement Fails: Detecting False Consensus via Hidden-State Consistency

Self-consistency (SC) decoding improves reasoning in large language models by sampling multiple chain-of-thought paths and returning the majority answer. Early-stopping variants halt sampling once answer agreement reaches a threshold, reducing cost but introducing a failure mode we term false consensus: the model may consistently produce the same wrong answer with high agreement. We propose CHSC-gated ESC, which augments agreement-based early stopping with a calibrated hidden-state consistency score (contrasting shallow vs. deep layers) and an optional adaptive threshold estimated online from running statistics. Correct-consensus paths exhibit more coherent hidden states than wrong consensus paths; the gap is strongest at shallow layers (Cohen’s d = 0.60, p < 0.001 at layer 3 on MATH). On GSM8K (n = 1,319) and MATH Level 4–5 (n = 500) with Qwen2.5-7B-Instruct, CHSC-gated ESC recovers ∼52% (GSM8K) and ∼29% (MATH) of the accuracy lost to false consensus under τa = 0.5, while retaining over 40% token savings vs. K = 8 sampling. Wilson intervals, conservative z-tests, and McNemar ranges from marginals appear in the main text and Appendix A.