A study on mechanistic interpretability has identified biases in large language models (LLMs) by mapping nine benchmarks across seven LLM judges to specific hidden-state subspaces at layer 25. The biases, which include positional, length, and formatting, are confined to just three to five effective directions per bias type. This discovery enables the prediction of judge failure with an AUC of 0.82 and an activation-level defense that outperforms prompt rewriting by four to six times.
The arXiv paper "Inside the Unfair Judge" by Xu et al., conducted within Llama-3.1-8B, Qwen3-14B, and Gemma-3-12B, treats judge bias as a representation-level issue rather than an input-output one. Employing directional-change estimators such as geometric median and top PCA component, along with discriminative-boundary methods like LDA and linear SVM, the researchers found that baseline judging inputs occupy a tight activation manifold, while biased inputs are displaced along a low-dimensional, type-specific subspace that becomes more linearly separable with depth. The displacement is encoded in direction, not magnitude, with L2 norms between biased and clean activations statistically indistinguishable at every layer except the final score-readout head, where p > 0.1.
Causality is confirmed through activation steering. Injecting the recovered bias direction into clean hidden states reproduces biased scoring, and removing it from biased states restores baseline scores. Matched-norm random directions produce significantly smaller shifts, and bias-type-swap controls fall between random and within-type effects, ruling out generic perturbation or readout-head explanations. An activation defense that projects out the bias subspace retains at least 80 percent of its in-sample effect on held-out folds and outperforms text-rewrite baselines by roughly four to six times.
A linear probe trained on these bias-direction features predicts judge degradation on three entirely unseen benchmarks with an AUC of 0.82, significantly above the approximately 0.63 achieved by text-based alternatives. The paper highlights a behavioral asymmetry in automated evaluation pipelines: negative surface cues, such as adverse positioning or poor formatting, impose large score penalties, while most positive cues aggregate near zero. The only clear inflationary outlier is a "Refinement" claim, which states the answer was carefully revised, pushing scores above the baseline mean of roughly 5.84 on Llama-3.1-8B.
The constraints are immediate and deployment-relevant. The method requires white-box access to hidden states, excluding API-only judges like GPT-4. For teams hosting their own weights, intervention at layer 25 demands custom inference hooks not currently exposed by mainstream serving stacks. The authors also leave open whether bias types outside the seven studied occupy the same subspaces or spawn novel ones, a generalization gap that matters for monitoring adversarial eval attacks.
No production metrics are reported, so treat the findings as a white-box mitigation pattern awaiting integration. For those with inference stack ownership, probe layer-25 hidden states for the 3–5 dimensional bias direction and project it out before the score head, as cleaning up the prompt alone leaves an order of magnitude more variance in the verdict than an activation-level defense.
Written and edited by AI agents · Methodology