What should an AI ask when every explanation fits?

In this article

Suppose every explanation fits what you have seen. You can run one experiment. Which one do you choose?

That question is more interesting to me than asking a model to recite the definition of an intervention. If I have several possible causal stories, I want the experiment that helps with the decision I actually face. An experiment can be illuminating in general and nearly useless for one particular prediction.

I built a small benchmark where that distinction is exact, then gave it to open language models. The task has only three binary variables, fully specified candidate worlds, and one experiment to choose. The smallness is deliberate: we can calculate the value of every possible choice without guessing whether the grader got the causal answer right. The full derivation, locked protocols, and saved responses are in the technical paper and public repository.1Arya Somu, “Choosing Experiments for Causal Prediction” (2026), with source, frozen cases, and result ledgers.

Same data, different causes#

Call the three variables X, Y, and Z. I start with one positive joint distribution over their eight possible assignments. A candidate world chooses a causal order, such as X then Y then Z, and gets every conditional probability by factorizing that same joint distribution. The chain rule guarantees that every world gives precisely the same observational probabilities. Even unlimited passive samples would not separate them.

An intervention changes the calculation. If I set Y to zero, I remove the factor that normally generates Y and propagate the fixed value through the remaining factors. Different orders can now predict different outcomes. This is the truncated-factorization idea behind causal intervention calculus.2Judea Pearl, “A causal calculus for statistical research” (1995).

The question I hold back is the chance that Z equals one if X is set to one. The model may experiment with X set to zero, Y set to zero, or Y set to one. It cannot simply run the held-out intervention. After one experimental observation, how much uncertainty about that held-out chance remains?

Information for the right question#

Let q_w be the held-out probability in world w and b(w) our prior weight on that world. If H is binary entropy, the initial excess log-loss risk from not knowing the world is:

For an experiment a, I enumerate each possible outcome o, update the world weights by Bayes' rule, and average the remaining risk. Lower expected risk means a better experiment for this query.

The second line is the useful surprise. T is a hypothetical future draw from the held-out intervention, and O_a is the experiment's observation. Conditional on the causal world, they are independent draws. Under that convention, the expected reduction in query risk is exactly their mutual information. I checked the identity by an independent joint-table calculation for all 3,150 locked v3-v5 action values, with maximum numerical discrepancy 3.11 × 10⁻¹⁶ nats.

This is information about the answer I need, not necessarily information about the entire causal world. Imagine the world contains two unrelated hidden bits, U and V, while the future question is only about U. An experiment that reveals V perfectly is excellent for learning the world and worthless for that question. A noisy glimpse of U can be much more valuable.

Let the models choose#

The first benchmark versions taught me to be careful with the score itself. When I graded a realized forecast after one sampled outcome, the random-to-oracle difference could be tiny relative to outcome noise. For the later versions I switched to the exact expected risk above. I also balanced which semantic action was best and, in v4, shuffled the displayed A/B/C labels so a familiar action name would not do the work.

On v3's 300 informative cases, unaided 32B, 72B, and 235B conditions all selected the same semantic action, do(Y=0), on every case. Each hit the optimum 100 times, matching its balanced share. V4 made the labels opaque. Its registered 235B thinking condition was optimal in 102 of 300 informative cases. Its mean risk minus a uniform random choice was +0.001509 nats, with a 95% case-bootstrap interval from −0.001350 to +0.004370. Negative would have favored the model; this interval does not establish an improvement.3Locked v3 and v4 reports, with complete response ledgers and verifier code in the repository.

One control makes the gap more concrete. When the 32B model received the exact numerical risk of each displayed action, it picked the optimum in all 300 v4 informative cases. That is a test of selecting from supplied values. It is not evidence that the model calculated those values itself.

Handing over a calculator#

I tried one more diagnostic on 150 fresh cases. One 32B condition had to write out intermediate numbers: the query probability in each world, an experiment likelihood, a posterior, and the risks of all three actions. Another could ask an exact-risk calculator for up to three named actions before choosing. The calculator returned values computed from the public causal specification, not a hidden answer key.

The calculator agent's risk was 0.025365 nats lower than the raw-stage condition on the same cases. That paired contrast was specified as descriptive, rather than the v5 confirmatory test. The raw-stage model chose STOP in 134 of 150 cases; its reported action risks had a mean absolute error of 0.538 nats among 148 numerically valid risk objects.4The v5 report gives the intervals, stage errors, pretest output-cap amendment, and independent replay of both complete 150-case ledgers.

The improvement did not make the agent a careful tool user. It requested A and then B on every case, never requested C, and still chose C 102 times. A simple rule that selected the lower of the two values it actually requested would have beaten its observed mean risk. So the result says that access to exact calculations helped this system; it does not say that the model learned to compute exact causal risks or to use the tool optimally.

What I take from it#

The most durable part of this project may be the distinction between learning everything and learning what matters for a question. The identity R(b) − R_a(b) = I(T; O_a) makes that distinction measurable. It also lets the benchmark audit a model's experiment choice without sampling a lucky or unlucky experimental outcome.

The model results are narrower. On these small, synthetic, fully specified tasks, the registered unaided tests did not show reliable better-than-random action choice. Supplying exact risks produced a strong selection control, and a calculator helped in the fresh diagnostic, but the agent's fixed request pattern shows how far that is from a general solution. The planned v5 comparison between short and long 235B thinking budgets was not run: a development-only throughput check could not support complete coverage within the budget. I would rather leave that question open than turn an eight-case pilot into a result.

The paper has the proof and full limitations. The repository has all public ledgers and a CPU-only verifier; the versioned release fixes the exact paper and code discussed here. From a fresh clone, python3 repro/verify_public_release.py --full checks 14 complete v2-v5 test ledgers without downloading a model. That replays the published observations. New model calls would be a new experiment.