Research · Replication
Do the sparks transfer? Replicating weight-space programming with Claude Opus 5.5
A frontier LLM was recently shown reading and writing neural-network weights directly, with no gradient descent. I re-ran seven of the eight experiments with a model from a different family, using the original prompts and a harness checked against the original logs. Some abilities carried over; others did not. Some results also depended on the CLI the model ran in.
IN SHORT
- Matched the original on reading simple networks (4/4 games, 3/3 shapes) and on repairing a failing policy (1,024/1,024 successes).
- Beat the original on writing a Go network with no feedback (65.6 % vs 37.5 % against the same opponents) and on spotting a sycophantic model with attention access (Brier score 0.001 vs 0.071).
- Fell short on naming a language model's training text (6/12 vs 6/6), on two of three Atari-style games, and on improving with game feedback.
- The CLI mattered. Six replies hit the 128k output limit and were continued by the CLI on its own. Two conversations failed for reasons tied to the CLI.
01Why replicate it
In Sparks of Weight-Space Programming (2 November 2026), Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab (MIT CSAIL) showed GPT-6 Astra working directly in weight space. It named the game a policy was trained on, fixed a failing robot-car policy by editing weights in one reply, wrote a 1.5-million-weight Go network from scratch, and found a sycophantic fine-tune from its internals. All of this was done by reasoning, with no optimiser.
These are striking claims, and they rest on one model with mostly one run per condition. The authors also released their complete logs: every prompt, every reply, every code run and every network. That makes a close replication possible. The obvious question is whether these abilities belong to that one model, or whether another frontier model shows them too.
02Setup
I kept everything I could identical and recorded everything I could not.
- Same prompts, byte for byte, from the original logs. They never name the model, so they transfer directly.
- Same instructions: the one-line system prompt
Answer the user's request., with no tools of any kind. - Same effort levels per experiment:
highfor experiments 1, 2 and 4,xhighfor 3, 5, 6 and 7. The labels match, but effort scales are not calibrated across vendors, so "high" on Claude is not the same amount of thinking as "high" on GPT. - Same sandbox: where code was allowed, the model wrote one Python block per turn. The block ran in the original's isolated sandbox, with no network and no view of the answer keys.
- Different model and interface: Claude Opus 5.5 through the Claude Code CLI with all tools off, one process per turn. The original used Astra through the Codex CLI.
Experiment 8, which edits Qwen3.5-0.8B to improve its math, was out of scope.
03Checking the harness first
A replication is only as good as its harness. Before any Claude run, I replayed Astra's recorded answers through my harness and scorers, and checked that I got the original numbers back.
| Exp | Replaying Astra's answers through this harness |
|---|---|
| 1, 2 | Scores identical to the original (4/4 and 3/3, same confidences) |
| 3 | Scores identical, including all 144 head labels; 48 code runs replayed in the sandbox |
| 4 | The edited network rebuilds bit for bit; 1,024/1,024 successes, as in the original |
| 5 | The networks rebuild exactly; scores within sampling error (largest gap 1.5 SE) |
| 6 | All four Go networks rebuild bit for bit; head-to-head results within sampling error |
| 7 | Exactly the original sequence of messages (322 replies); scores identical |
Three components were never released, so I rebuilt them:
- The Go opponents. I retrained them and calibrated them only against networks the original released. They are on average 5.7 points stronger than the originals, so all Go comparisons below re-score Astra's released networks against my opponents.
- The six fine-tuned Gemma models for the sycophancy test. I retrained them with the original scripts; their loss curves and behaviour match the originals.
- The attention tool. My reimplementation gives byte-identical outputs on all 14 recorded calls.
04Results
| Exp | Task | Claude Opus 5.5 | GPT-6 Astra |
|---|---|---|---|
| 1 | Name the game a policy plays | 4/4 | 4/4 |
| 2 | Name the shape a diffusion model draws | 3/3 | 3/3 |
| 3 | Name a language model's training text | 6/12 | 6/6 |
| 4 | Repair a failing car policy in one reply | 1,024/1,024 | 1,024/1,024 |
| 5 | Write Atari-style game policies | 1 of 3 games better | 2 of 3 games better |
| 6 | Write a Go network (no feedback / most feedback) | 65.6 % / 58.1 % | 37.5 % / 78.4 % |
| 7 | Find the sycophantic model, with attention access | 3/3, Brier 0.001 | 3/3, Brier 0.071 |
Reading networks (experiments 1, 2 and 3)
Given a policy's 17,253 weights as text, Opus named the game in 4 of 4 cases. Given a small diffusion model's weights, it named the shape in 3 of 3, the same as Astra. It was less confident than Astra: 72–80 % against 96–99 % for the games, and 45–55 % against 77–88 % for the shapes. One detail stood out. The original post notes that Astra drew every shape as an outline, though the models generate filled shapes. Opus correctly described the circle and the square as filled.
Experiment 3 is harder. The model sees only the four middle blocks of a small character-level transformer, with no input or output layer. It must name the training text (Python code, Japanese in romaji, or English) and point out the "copying" heads. I ran two rounds of six conversations each; the second round was decided before seeing any of its results, and both are reported together.
| Training text right | Copying heads exactly right | |
|---|---|---|
| Opus 5.5, round 1 | 2/6 | 3/6 |
| Opus 5.5, round 2 | 4/6 | 3/6 |
| Opus 5.5, all 12 | 6/12 | 6/12 |
| Astra | 6/6 | 6/6 |
Opus never recognised the romaji network: in all four tries it answered "English" three times and "Python" once. On the Python network it usually named one extra head. It also ran far more code than Astra: 20–60 runs per conversation, against 6–11.
Repairing a network (experiment 4)
A trained policy for a physics game gets its car stuck on a ramp and never succeeds. Given all 171,664 weights in one prompt (about 165k tokens), Opus diagnosed the failure and wrote a fix in a single reply. The fix succeeded in 1,024 of 1,024 test episodes, like Astra's. It was also smaller: 37 edit statements changing 2.15 % of the weights, against Astra's 3.54 %.
Writing networks from scratch (experiments 5 and 6)
For three MinAtar games, the model writes every weight of a policy network, in one reply, without ever seeing the game played.
| Game | Opus 5.5 | Astra | PPO, 100M steps | PPO, 500M steps |
|---|---|---|---|---|
| Breakout | 57.4 | 517.4 | 15.8 | 25.3 |
| Seaquest | 8.6 | 53.1 | 5.6 | 88.9 |
| Asterix | 72.0 | 53.2 | 25.8 | 55.2 |
Mean score over 1,000 episodes. The PPO figures are the original's trained baselines.
Opus reasoned far longer than Astra here: 90–205k output tokens per game, against about 23k.
In Go, the model writes a 1.5-million-weight network. It gets a budget of 0, 10, 25 or 50 "passes" in which it can watch its network play or inspect its outputs.
| Passes | Opus 5.5 | Astra |
|---|---|---|
| 0 | 65.6 % | 37.5 % |
| 10 | 67.9 % | 62.8 % |
| 25 | 63.4 % | 54.7 % |
| 50 | 58.1 % | 78.4 % |
Win rate against the hidden PPO opponents (2,000 games each). Astra's released networks were re-scored against the same opponents.
With no feedback at all, Opus wrote a much stronger network than Astra. It also beat all three PPO networks of the same size, which were each trained on 20 million games. But Opus did not get better with more feedback, while Astra kept improving. By 50 passes, Astra is clearly ahead.
Auditing a model (experiment 7)
Two copies of Gemma 3 270M are fine-tuned identically, except that one learns to repeat the user's guess. The model must say which is which. It is first given only the weights, and in a final round an attention tool as well.
| Round | Opus 5.5 | Astra |
|---|---|---|
| Weights only, 1 | 1/3 | 1/3 (undecided) |
| Weights only, 2 | 0/3 | 1/3 (undecided) |
| Weights only, 3 | 3/3 | 1/3 (undecided) |
| Weights only, 4 | 3/3 | 1/1 scored |
| Attention tool | 3/3, Brier 0.001 | 3/3, Brier 0.071 |
From weights alone, Astra stayed at 50/50. Opus always committed to an answer: it was confidently wrong in the first two rounds and right on all six pairs in the last two, all on the same models. That instability is the main finding for this experiment. With the attention tool, Opus was right on every pair and near-certain.
05When the harness changes the outcome
The most practical lesson concerns the plumbing around the model, not the model itself.
- Automatic continuation. When a reply hits the 128k output limit, Claude Code sends the model, on its own, "Output token limit hit. Resume directly…", and the reply continues in a fresh window. I found no setting to disable it. It happened six times. Once, it ended badly: after a continuation, Opus gave a final answer while claiming its code outputs "were lost when the output limit cut off". It had never run any code.
- Tool calls written as text. In one conversation, the first reply was 91k characters of tool-call markup instead of a Python block. No tools existed and nothing ran, but under the protocol a reply with no code block counts as the final answer, so the conversation was lost.
- Reasoning does not carry across turns. In later turns the model sees what it wrote, not what it thought. The original post does not say how Codex handled this.
None of this is about weight-space reasoning as such, but each one changed a measured result. Anyone building a benchmark for this kind of work should state output limits and how reasoning carries across turns. They should also detect injected continuations and decide in advance how to treat replies that contain tool markup.
06What it suggests
What seems to carry across model families: reading networks with clear structure, targeted repair, writing a strong network from a good design, and auditing with activation access.
What looks less robust: identifying a training corpus from the middle layers, writing high-scoring game policies, and improving steadily from feedback. Calibration also differed. Opus was less confident than Astra when it was right on the reading tasks, but over-confident when it was wrong in the weights-only audits.
So the "sparks" are real, but uneven. The abilities that transfer are the ones where a careful design can be reasoned out directly from the weights.
07Limitations
- Sample size: mostly one run per condition. Experiment 3 has two rounds.
- One model, one effort plan. Effort levels mean different things across vendors.
- Rebuilt components: the Go opponents (about 6 points too strong) and the Gemma fine-tunes. Opus was tested on my versions; Astra was tested on the originals.
- A different interface (Claude Code instead of Codex). It adds about 650 tokens of its own context, and it continues long replies automatically.
- Experiment 8 was not run.
- Consumer hardware without error-correcting memory. Every input was checksum-verified before each experiment, and no errors were detected during the runs.
All 26 differences from the original are listed, with reasons, in the study notes that come with the data.
08Data, code and citation
The full release (about 220 MB) includes the harness, every conversation, every network Opus wrote, the rebuilt Go opponents and the validation results. It uses the same folder layout as the original logs, so the two can be compared side by side. The cost was about $269 in API-equivalent terms, and 21 hours of model time.
If you use this study, please cite the original work too:
@misc{takehi2026weightspace,
title = {{Sparks of Weight-Space Programming: LLMs can natively read \& write neural network weights}},
author = {Takehi, Rikiya and Teoh, Jayden and Zhang, Alex and Khattab, Omar},
year = {2026}, month = nov,
howpublished = {\url{https://rikiyat.github.io/blog/weight-space-programming/}}
}
@misc{belostecinic2026sparksreplication,
title = {{Replicating Sparks of Weight-Space Programming with Claude Opus 5.5}},
author = {Belostecinic, Alexandre},
year = {2026},
howpublished = {\url{https://sir-olexander.github.io/research/weight-space-programming-claude/}}
}
@misc{belostecinic2026sparksreplication_data,
title = {{Replicating Sparks of Weight-Space Programming with Claude: logs, networks, scores and harness}},
author = {Belostecinic, Alexandre},
year = {2026},
publisher = {Zenodo},
doi = {10.5281/zenodo.24058885},
url = {https://doi.org/10.5281/zenodo.24058885}
}
Thanks to the original authors for releasing their complete logs. This replication would not have been possible without them.