MIT researchers report that GPT-6 Astra repaired repetitive reasoning in a small Qwen model by directly editing its weights, the numbers that control its responses. MATH-500 accuracy rose from 7.9% to 48.1%. Before the edit, 93% of MATH-500 responses exhausted the experiment’s output allowance; afterward, 33.1% did.
Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab, all listed at MIT CSAIL, released the findings on November 2 in a blog post, Sparks of Weight-Space Programming. Their report describes a frontier model inspecting and changing another network’s numerical weights without gradient descent. The large Qwen gain came from a targeted repair to repetitive reasoning.
The experiment gives developers a specific repair to examine: an edit that helped an existing model finish answers. The released records describe repeated tests using practice answers and a failed follow-up on another model. The strongest Qwen run took about 20 hours.
What Astra changed
Weights are the numbers that determine how a neural network transforms its input. Training usually adjusts those numbers using gradients, which indicate how a small change affects an error measure. Here, Astra used a Python workspace to inspect Qwen3.5-0.8B’s weights and propose changes directly. Its editing workspace had no GPU or internet connection; a separate tool ran the model and returned measurements.
The strongest run used 50 successful feedback calls, 80 Python runs and 158 turns. A feedback call could generate up to 512 answers or inspect internal measurements from up to 1,024 texts. Astra could compare original and edited models on the same practice problems, examine failures and revise its edit. The run lasted 20.3 hours, including a 4.7-hour service outage.
Astra eventually altered selected output columns in two feed-forward layers, numbered 15 and 19. These layers mix internal signals that help determine what text the model generates. The patch counteracted directions associated with repetition, touching 131,072 weight values, about 0.016% of the stated 0.8 billion. After conversion to the lower-precision format used for evaluation, 130,188 values actually differed from their originals.
More finished answers, under a fixed cap
Both models used thinking mode, temperature 0.6 and the same 32,768-token allowance for reasoning and an answer. The scores below average individual responses, including failures. MATH-500 uses four responses per problem; its 48.1% is not the share solved at least once in four tries.
Reported benchmark results after the weight edit
| Evaluation | Original | 50-call edit |
|---|---|---|
| MATH-500 | 7.9% | 48.1% |
| GSM8K | 13.8% | 67.1% |
| IFEval, strict prompt-level | 21.0% | 62.3% |
| AMC23 | 0.5% | 23.4% |
| Minerva Math | 1.5% | 25.4% |
| OlympiadBench | 1.3% | 21.8% |
| MMLU-Pro, 1,000 questions | 40.3% | 45.5% |
| GPQA Diamond | 22.7% | 36.6% |
| AIME 2024 | 0.0% | 0.6% |
| AIME 2025 | 0.0% | 2.3% |
Percent correct responses under the researchers’ settings; different benchmarks use different response counts. MATH-500 averages four responses per problem.
Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab, completed experiment results, CC BY 4.0. SnowRock checked the released score totals.
The blog’s GPQA figure uses an interim baseline of 24.5%; the completed evaluation records 22.7%. MATH-500 rose by 40.2 percentage points, roughly sixfold. Instruction following on IFEval nearly tripled, while the two AIME scores remained low.
The agent’s own confirmation on 1,152 held-out practice problems supports the loop diagnosis. It recorded 579 answers changing from wrong to right and five changing the other way. Of those rescued answers, 576 had originally exhausted the token allowance. Those practice results are separate from the benchmark table.
Qwen’s official model card already warns that the 0.8B model can enter thinking loops. Its current guidance recommends temperature 1.0 for thinking on text tasks and up to 81,920 tokens for complex benchmarks. The researchers held their chosen settings constant for the comparison. Testing the patch against a better-tuned original model remains necessary.
What remains unproved
The experiment used practice data with answers. A preparation script filters normalized duplicates and shared 13-word sequences against evaluation questions. That addresses direct overlap in the supplied practice material. It cannot establish what either model encountered during earlier training. The public record also omits earlier prompt trials and does not provide repeated independent runs establishing a reliable success rate for discovering this patch.
The team also edited Gemma 3 270M, whose original responses hit its 8,192-token cap only 8.5% of the time. With format-tolerant grading, MATH-500 scored 6.8% for the original and 7.0% after 50 successful feedback calls; earlier-budget runs reached at most 7.7%. The authors report no established improvement. Gemma used different sampling settings, model size and grading, so its scores cannot be compared directly with Qwen’s.
Direct weight editing has a research history. The ROME study, presented at NeurIPS 2022, changed factual associations through targeted feed-forward weight updates. This experiment instead puts a general-purpose language model in charge of inspecting internals and designing a repair.
The authors release weight differences and a script that checks reconstructed tensors against recorded hashes. The full software setup used to conduct the math-editing experiment and some grading dependencies are absent. SnowRock reviewed the released records without rerunning the models. Independent tests of the frozen patch, including a comparison against a better-tuned original, would establish how far this repair carries beyond the reported run.
