From the paper Training Vision Models for LLM-Driven Weight-Space Programming · Ropinski & Kranz · Ulm University

Interactive Weight-Space Programming Agent Wall

1,986 recordings of LLM agents editing vision models directly in their weights: no gradients, no fine-tuning, only probes, weight reads and weight writes through our WeightLab tools.

1,986 trajectories5 LLM editors3 backbones4 datasets2,600+ scored edits in the paper
Hover a cell for its facts, click to open it, scroll or pinch to zoom, drag to pan. In an open trajectory: click a tick (or ← →) to play that one step and hold on its result; Play continues. inspect backbone edit, source row of the head target row of the head, success failed or rejected call

Video

Narrated overview of the paper (2:43): Edit-Ready Training, WeightLab, results, and the trajectory wall.

BibTeX

@misc{ropinski2026training,
  title         = {Training Vision Models for LLM-Driven Weight-Space Programming},
  author        = {Ropinski, Timo and Kranz, Sebastian},
  year          = {2026},
  eprint        = {2611.26408},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV}
}

The paper in brief

Each agent receives an edit request (relabel a concept, fix a confusion between two classes, or fix a misclassified subgroup) with a few support images. It may change any weight, under a budget of 20 probes, 30 reads and 8 edits, and never learns how the network was trained. Every edit is scored on held-out images as success minus side effect.

Agents mostly write sparse, rank-one edits to the classifier head. Edit-Ready Training (ERT) rehearses exactly such edits during training and backpropagates through them, so the features become easy to edit. ERT gives the best mean net score for both frontier editors: GPT-6 Astra 0.74 (standard training 0.63) and Claude Opus 5.5 0.72 (0.60), with the largest gains where standard networks are hard to edit.

Overview: (a) Edit-Ready Training simulates edits during training; (b) an LLM edits the network through WeightLab; (c) mean net edit score per training method.

(a) ERT simulates edits during training. (b) An LLM edits the network through WeightLab. (c) Mean net edit score over all nine settings.