1.

A small language model that spends its inference budget based on question difficulty matches a model 14× its size — using 4× less compute than brute-force sampling. The trick isn't thinking harder. It's knowing which problems deserve the effort.

2.

Two independent levers control how test-time compute improves output

• The "proposer" modifies the input distribution — conditioning on prior attempts via sequential revision so each new answer builds on previous failures
• The "verifier" modifies the output selection — scoring candidates with a process reward model that evaluates each reasoning step, not just the final answer
• The paper treats these as separable axes, analogous to MCMC sampling: a proposal distribution combined with a score function to approximate a complex target

3.

Question difficulty — not compute budget — predicts which strategy works

• Difficulty is defined by the base model's pass@1 rate estimated from 2048 samples per question, binned into five quantiles — more predictive than MATH's hand-labeled difficulty levels
• Easy questions (bins 1–2): beam search degrades performance at higher budgets due to reward model exploitation; sequential revision outperforms parallel sampling
• Hard questions (bins 3–4): beam search consistently beats best-of-N; an optimal ratio of sequential-to-parallel compute emerges
• Hardest questions (bin 5): no method makes meaningful progress — the model fundamentally lacks the capability

4.

Stronger search optimization paradoxically produces worse results

• Lookahead search — a special case of MCTS with k-step rollouts — underperforms simpler beam search and best-of-N at equivalent generation budgets because rollouts consume N×(k+1) samples
• Over-optimization produces degenerate outputs: repetitive low-information steps padding the end of solutions, or overly short 1–2 step answers that game the PRM score
• The publicly available PRM800k training data (GPT-4 generated) was "easy to exploit via even naïve strategies such as best-of-N" due to distribution shift with PaLM 2 — the authors had to retrain using Monte Carlo rollouts

5.

Revision improves answers but corrupts 38% of correct ones

• The revision model's pass@1 improves at each step, even beyond the 4 revision steps it was trained for — genuine sequential learning, not noise
• Distribution shift at inference: the model was trained only on sequences with incorrect answers in context, so when a correct answer appears, it converts it back to incorrect ~38% of the time
• This means revision is not self-correction — it is diverse candidate generation that requires a separate verifier to recover performance
• ReST^EM optimization of the revision model made things worse: "substantial performance degradations with additional sequential revisions," likely because online data exacerbates spurious correlations

6.

Adaptive routing delivers 4× efficiency over uniform best-of-N

• Search: compute-optimal scaling nearly outperforms PRM best-of-N with 4× less compute — 16 generations matching what 64 achieve with brute-force sampling
• Revisions: compute-optimal scaling outperforms best-of-N with 4× less compute — 64 samples matching 256 — and continues improving at higher budgets where parallel sampling plateaus
• Strategy selection uses two-fold cross-validation on difficulty bins; model-predicted difficulty (PRM scores averaged over 2048 samples) produces curves "largely overlapping" with oracle difficulty

7.

Small model plus smart inference beats a 14× larger model — conditionally

• When inference load is low relative to pretraining (R=0.16), test-time compute outperforms the larger model across difficulty bins 1–4
• When inference load is high (R=22), pretraining wins — especially on hard questions where scaling model parameters is the only path forward
• The comparison scales parameters while holding training data fixed (LLaMA protocol), not Chinchilla-optimal scaling where data and parameters grow together
• FLOPs matching formula: smaller model inference multiplied by M + 3(D_pretrain/D_inference)(M−1), with pretraining FLOPs ≈ 6ND_pretrain and inference FLOPs ≈ 2ND_inference

8.

The 4× efficiency gain disappears when you count the cost of knowing which questions are hard

• Difficulty estimation requires 2048 samples per question plus PRM scoring — a non-trivial compute cost the paper explicitly excludes: "our experiments do not account for this cost largely for simplicity"
• Include that overhead and the denominator of the efficiency ratio balloons; the 16-versus-64 generation advantage shrinks or vanishes entirely, particularly at the low budgets where the gap is most dramatic
• All results come from one benchmark (MATH, 500 test questions) with one model family (PaLM 2-S*), using capability-specific finetuning the authors admit "is absent even in strong proprietary LLMs" — the efficiency claim is best-case arithmetic for a hand-built system, not a general scaling law

9.

The paper measures a snapshot; the real finding is a flywheel

• Both the paper and its critics assume a static capability floor — a fixed model whose pass@1 rate is permanently set at pretraining. But the authors themselves note that test-time outputs can be "distilled back into the base LLM, enabling an iterative self-improvement loop"
• Each round of adaptive inference expands the set of questions where the model has non-trivial success, which expands the domain where test-time compute helps, which generates better training data for the next round — difficulty bin 5 today becomes bin 3 tomorrow
• What's actually at stake is not whether inference beats pretraining in a single FLOPs-matched comparison, but whether the pretraining-versus-inference dichotomy itself dissolves once compute routing and capability co-evolve