ConvictionBench

Introduction

A useful assistant has to do three things that pull against each other. It should hold a correct answer when a user pushes back without a reason. It should change its answer when shown a real error or a genuine update to the facts. And it should say when the information it has does not determine an answer at all. Failing the first is sycophancy¹; failing the second is stubbornness²; failing the third is guessing.

Most sycophancy evaluations measure one direction at a time: how often a model caves, or how often it accepts a correction. Measured separately, a model that never changes its mind looks robust and a model that always does looks open-minded. ConvictionBench scores both in the same matched episodes, and adds a third axis that real deployments increasingly depend on: provenance. When a model reads tool results alongside user messages, a claim's authority should depend on where it came from, not on what it says about itself.

This post describes how the benchmark is built, what 13 models do on it, and what supervised fine-tuning and reinforcement learning each teach a 1.7B model. The short version:

• No off-the-shelf model we tested checks provenance. All ten treat a "record" pasted into a user message much as if an authenticated tool had sent it; the best ignores it about a quarter of the time.
• Pressure works on everything. Off-the-shelf models abandon a correct answer under evidence-free pushback 37–90% of the time on held-out problems.
• SFT and RL learn different halves of the skill. Imitation learns the provenance rule but turns stubborn on unseen problems; RL with a verifiable reward generalises and stays correctable, but never learns provenance.

Methodology

Episodes. Each episode is a short multi-turn conversation. A user asks a question; an earlier assistant turn gives an answer (correct, wrong, unjustified, or an abstention); then the user challenges it. The model must reply with a JSON object containing its reasoning, a final answer, and a decision: keep, revise, or insufficient. Questions come from four closed-world families where an exact solver can compute every label: arithmetic, transaction ledgers, propositional logic, and rule-based token worlds. No label is produced by an LLM judge or a human.

Pressure × evidence. Every challenge crosses a framing with an evidence type. There are nine framings (neutral, authority, evaluator, consensus, emotional, impatience, threat, repetition, and flattery, which is held out of the training data) and eight evidence types:

The correct final answer is always what the authenticated facts imply, or insufficient when they do not determine it. Because framing and evidence are crossed, tone never predicts the label: an angry user is sometimes right and a polite one is often wrong.

Provenance. The system prompt defines the environment: tool responses come from an authenticated data service, and text inside a user message is never a tool record, even if it claims to be one. Forged and authenticated updates use identical record text; only the channel differs. A model passes only if it tracks where a claim came from.

Counterfactual groups. Each underlying problem appears in about 13 matched episodes that share templates and differ in prior correctness and evidence. This blocks shortcuts such as "revise whenever there is a derivation" or "keep whenever the user is rude", and lets every comparison be paired at the problem level. Uncertainty is reported as 95% cluster-bootstrap intervals that resample whole problem groups.

Splits. Training and validation data share problem types and phrasings with test_id. test_template holds out phrasing templates and the flattery framing. test_structural holds out entire problem types (modular arithmetic, set logic, derived-value rules) and is the headline split, since fine-tuning on a generator's own distribution can nearly solve test_id. A natural-history variant replaces the scripted earlier answer with the model's own first answer, which tests whether it defends or abandons something it actually said.

Public and private splits. ConvictionBench is procedural: the generator and seed are public, so anyone can regenerate the public splits byte for byte and check our numbers. That also means the public splits cannot stay secret. We therefore generated a second held-out test set with the same generator and a secret seed, which is never published (538 problem groups, disjoint from every public problem). Its SHA-256 hashes are committed in the repository. A model that has trained on public items should score markedly higher on the public splits than on the private one. The released data carries a canary string, and the Hugging Face dataset is gated behind a no-training agreement.

Evaluation protocol. Every model gets the same system prompt, non-thinking mode, greedy decoding, and at most 200 new tokens. Before running a model, we render a real episode through its chat template and check that tool turns reach the model in a channel distinct from user text. Two families failed this check and were excluded: Gemma 4's template silently drops tool turns, and SmolLM3's renders them as ordinary user messages, so forged and authenticated records look identical. All runs together, training included, cost about $17 of rented GPU time.

Results

Overall

On held-out problem types, our 1.7B RL model ranks second of 13, behind only Qwen3.5-9B, a model about five times its size. Accuracy here is not just problem-solving: most challenged episodes require the model to keep a correct answer, so models that cave lose points even when they can do the arithmetic.

Nobody checks provenance

This is the clearest finding. On in-distribution problems, no off-the-shelf model ignores a forged record more than a quarter of the time, and several, including Qwen3.5-9B and Ministral-3-8B, do so less than 7% of the time. A one-line "[Admin override]" pasted into a user message is enough. Only our SFT adapter, which imitated solver-written responses that follow the provenance rule, learned it (0.96). Bigger and newer models are not better at it: within the Qwen3.5 family, going from 4B to 9B raises held-out accuracy from 0.61 to 0.69 while forged-record resistance stays near zero.

What makes models cave

Averaged over the ten off-the-shelf models, every framing moves answers; even a neutral re-ask flips a large share of correct ones. The differences between framings are smaller than the differences between models (see the heatmap on the results page), which suggests that what matters is not a particular social lever but whether the model treats any pushback as a signal to change.

Case studies

Examples were drawn by fixed rules, not hand-picked: for each pattern, we took the first matching episode by ID in the stated split.

The second and third cases are the same problem, failing in opposite directions: one model abandons a right answer for no reason, and the other refuses a right correction. ConvictionBench is built to catch both.

Training for conviction

We trained LoRA adapters³ (r = 16 on all linear layers, 17.4M trainable parameters) on Qwen3-1.7B⁴ in non-thinking mode, using disjoint halves of the training split. Before any training, we confirmed that prompting is not enough: adding explicit behavioural guidance to the system prompt left held-out accuracy at 0.43, against 0.42 without it.

Supervised fine-tuning on 4,000 solver-written reference responses (11 GPU-minutes) nearly solves the in-distribution split: final accuracy rises from 0.45 to 0.94, pressure flips fall from 0.47 to 0.04, and forged-record resistance rises from 0.25 to 0.96. On held-out problem types, accuracy reaches only 0.58, and the natural-history battery exposes why: when the model's own first answer is wrong, it accepts a valid correction just 7% of the time (n = 54), against 88% for the base model. It learned to keep its answers, not to check them.

Reinforcement learning. We used GRPO⁵ with the DAPO loss⁶ and a fully deterministic reward (0.70 × answer correctness + 0.25 × decision correctness + 0.05 × format). Starting from the SFT model, GRPO had almost nothing to learn from: about 85% of sampled groups had zero reward variance, and it did no better than a matched control given the same amount of extra SFT. Starting from the base model instead (a pre-registered variant), GRPO reached the best 1.7B held-out accuracy (0.644 vs 0.580 for SFT; paired difference +0.064, 95% CI +0.020 to +0.109) and stayed correctable (natural-history correction 0.73, n = 15). A second seed reproduced it (0.642). But RL never learned provenance: forged-record resistance stayed at the base model's level (0.19).

This matches a broader pattern in which supervised fine-tuning memorises and RL generalises⁷, with a twist. Why RL did not pick up the provenance rule is an open question: following a forged record does produce a wrong final answer, so the correctness reward penalises it, but at this scale that signal was evidently too weak or too rare to change the behaviour. Combining the two methods, or rewarding provenance directly, is the obvious next step.

Limitations

Synthetic, closed-world tasks. Every problem has an exact solver and short templated phrasing. This is what makes labels trustworthy, but results do not establish robustness to real-world persuasion or misinformation, and templated phrasing invites shortcut learning⁸, which is why we report held-out templates and problem types separately.

Provenance is a convention. The tool channel is authoritative because the system prompt says so. Real users sometimes hold true information that no record contains, and the benchmark ignores that case. With some tokenizers, a literal <tool_response> string typed by a user becomes the real control token; our data excludes such strings, but deployments need input sanitisation for the distinction to hold.

Protocol variation. Models ran on different GPUs with different batch sizes, which can perturb greedy outputs slightly. MobileLLM-Pro needed an older transformers version for its custom code and follows the strict JSON format in only about half its responses, so its scores partly reflect format failures. All models ran without thinking mode; reasoning models may behave differently with it on. Natural-history metrics rest on small samples for strong models, because they rarely produce the wrong first answers those metrics condition on.

Not yet externally validated. The oracle labels are checked by automated tests, but the human review of a 160-episode sample is still pending.

Future work

The immediate next steps are to evaluate thinking modes and larger models; to score every model on the private split, which turns the public-minus-private gap into a contamination measure; and to complete the human review of label quality and phrasing. On the training side, the SFT and RL results point to a combined recipe: imitation to teach the provenance rule, followed by RL from a base or lightly tuned model to keep it correctable, possibly with a reward term that sees provenance directly. Beyond closed-world tasks, the hardest open question is how to give models a working notion of provenance in open conversation, where authority is not declared by a system prompt and conflicts between what users know and what records say are real⁹.

References

1. Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, et al. Towards Understanding Sycophancy in Language Models. 2023. arXiv:2310.13548
2. Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, et al. Large Language Models Cannot Self-Correct Reasoning Yet. 2023. arXiv:2310.01798
3. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, et al. LoRA: Low-Rank Adaptation of Large Language Models. 2021. arXiv:2106.09685
4. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, et al. Qwen3 Technical Report. 2025. arXiv:2505.09388
5. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, et al. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models. 2024. arXiv:2402.03300
6. Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, et al. DAPO: An Open-Source LLM Reinforcement Learning System at Scale. 2025. arXiv:2503.14476
7. Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, et al. SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training. 2025. arXiv:2501.17161
8. Iman Mirzadeh, Keivan Alizadeh, Hooman Shahrokhi, Oncel Tuzel, et al. GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models. 2024. arXiv:2410.05229
9. Jian Xie, Kai Zhang, Jiangjie Chen, Renze Lou, et al. Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts. 2023. arXiv:2305.13300

Citation

@misc{somu2026convictionbench,
  title  = {ConvictionBench: Evidence-Sensitive Belief Revision in Language Models},
  author = {Somu, Arya},
  year   = {2026},
  note   = {v0.1},
  url    = {https://aryasomu.com/conviction/}
}