How Astra Edited Qwen’s Weights

Weight-space programming means inspecting and directly modifying the numerical parameters of a neural network. Instead of asking a training optimizer to find an update, an agent can write code that measures internal behavior, constructs a specific change, and tests the resulting model. The weights…

Weight-space programming means inspecting and directly modifying the numerical parameters of a neural network. Instead of asking a training optimizer to find an update, an agent can write code that measures internal behavior, constructs a specific change, and tests the resulting model. The weights retain that change when the next request arrives.In the November 2026 Astra experiments, the released checkpoint, edit manifests, and investigation logs make this proposed model-repair workflow inspectable: we can connect the claimed improvement to the actual numbers changed. The releaseAstra, the AI agent conducting the investigation, edited Qwen3.5-0.8B, a small language model. It modified another model’s weights, not its own.A persistent weight change is reused at every generation step. These paths illustrate the relationship, not recorded text.How could changing a small part of Qwen’s weights produce a large improvement?The compact answer is that Astra changed what selected internal components contribute to Qwen’s working representation. Those contributions depend on the current input. The edited model consequently follows different generation paths, often escaping repetitive reasoning that previously consumed its entire output allowance.The reported MATH-500 accuracy rose from 7.9% to 48.1%. Those figures average four generations for each of 500 questions; they are not the percentage of questions solved at least once. Under the same study settings, the share of generations hitting the 32,768-token limit fell from 93.0% to about 33.1%. Released evaluation resultsReported MATH-500 results: 500 questions, four samples each. Thinking enabled; temperature 0.6, top-p 0.95, top-k 20, presence penalty 1.5, cap 32,768 tokens. Correctness and cap-hits can overlap; these are not new measurements.This gives us a useful first distinction. A math benchmark measures whether a run produces a correct, extractable answer under its evaluation conditions. A model that repeatedly restarts its reasoning can fail that test even when parts of its reasoning are useful. Changing how it continues can improve the measured score without establishing that new mathematical knowledge was inserted.Accuracy and completion are also separate outcomes. A run can write the correct answer and then continue until the limit. We need both measurements, plus the actual intervention, to understand the result.We will follow the computation from MLP columns to changed generations, then examine the controls, alternative generation settings, and requirements for reproduction.ContentsWhat weight-space programming changesHow an MLP writes into a token stateHow Astra found a candidate repairWhat the final patch computesWhy generations changedWhich conclusions survive the controlsReproduce the patch and test the claim Read more

Source: The AI Edge — Published November 9, 2026 at 16:01 UTC — Category: Research