04 Beyond inspection
Answers no amount of looking determines
The wheel lists all 30 families; the cards around it are real instances with their exact answers.
Some visual questions can't be answered by looking. A frozen model must write and run a real visual algorithm — the program becomes the reasoning — and it teaches itself which algorithm to write.
Thinking-with-images uses tools to expose visual evidence, then reasons in language. Some questions leave no gap a better view can fill — they require running an algorithm over the pixels.
Some visual questions have exact answers that no amount of looking reveals — you have to run an algorithm over the pixels. Give a frozen model only a Python interpreter and the program becomes the reasoning. Then let it teach itself which program to write: a training-free loop turns its own failed code into portable skills.
30 task families, answers beyond inspection, labeled by construction — no cap, no human annotation.
The model reads (and re-runs) its own failed programs to grow a small library of portable skills. No gradients.
An interpreter beats extended thinking; skills add 19–24 more points; they transfer across scale and family.
A frozen model solves each task by Code-with-Image. Failed trajectories drive two reflection modes, and a delivered skill ships only if it beats the bare solver on a held-out split.
The image enters a Python sandbox as data. The model writes a program that computes the answer — running it is the reasoning.
It reads its own failed attempts and edits a skill library; when stuck, it re-runs the failures in the sandbox and checks repairs against gold.
A skill ships only if it beats the bare solver on a held-out split, then rides along in the frozen model's prompt — plain, transferable text.
Each family is a program that constructs an image from a known latent target, so the label is correct by construction and instances can be generated without bound. No reference toolchain exists to imitate.
The wheel lists all 30 families; the cards around it are real instances with their exact answers.
An interpreter lifts accuracy where thinking barely moves it; self-evolved skills add another large step; and the skills are plain text that transfers across scale and family.
On questions that require running an algorithm, extended thinking barely helps, while a bare Python interpreter lifts every executor from 8B to frontier scale.
→ The bottleneck is the medium, not the chain length.A training-free reflection loop turns the model's own failed programs into portable procedures, adding another large step on top of the interpreter.
→ Knowing which algorithm to write is learnable, without gradients.Paired ablations attribute a quarter to a third of the total skill gain to executable reflection — the faults that never print require re-execution to see.
→ Debugging code becomes debugging reasoning.If you find Code-with-Image or CwI-Bench useful, please cite the paper.
@article{yang2026cwi, title = {Self-Evolving Code-with-Image Reasoning}, author = {Yang, Tianze and Wu, Liang and Sun, Ruitong and Shi, Yucheng and Wang, Yanqiao and Darbari, Mayank and Liu, Ninghao and Sun, Jin and Hong, Liangjie}, journal = {arXiv preprint}, year = {2026}, url = {https://github.com/YangTianze009/LVLM_POT} }