🔍 Read the full analysis: The Qwen Weight Edit That Improved GPT-6 Astra’s JSON Tab Escaping Results on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
An experiment reported by ThorstenMeyerAI.com found that a rank-one edit to one Qwen3-4B-Instruct-2507 weight matrix raised exact answers on a frozen JSON-escaping test from 49 of 120 to 70. The gain was limited to a narrow synthetic task: table-row extraction remained at zero of 30, and the edit has not had an independent review.
A GPT-6 Astra agent improved a small Qwen language model’s exact answers on a frozen test of copying text into JSON, according to an experiment published by ThorstenMeyerAI.com. After a rank-one edit to one matrix, Qwen3-4B-Instruct-2507 answered 70 of 120 cases correctly, up from 49; the result applies to a narrow synthetic task and has not been independently reviewed.
The experiment asked the model to place short text containing tabs and other special characters inside a JSON object. A response counted as correct only when it was valid JSON and, after decoding, reproduced the original text exactly. In this format, a tab inside a string must be represented with the escape sequence \t; copying a raw tab or changing the text made the answer incorrect.
The report says GPT-6 Astra, running as a Codex agent, planned and conducted the experiment. It inspected the model’s weights and calculated a rank-one update to the output projection of the final multilayer perceptron layer. The report says no gradient training was used and that the other 397 stored tensors were bit-identical to the original model.
On the frozen final test, the edited model got 21 more cases right and broke none of the answers the original got right, according to the report. The 21-case difference corresponds to 17.5 percentage points of the 120-case test. The original model and a version given a more explicit escaping prompt each scored 16 of 32 on an earlier screening set; that screening result is separate from the final test.
Infographic 01 · The measured result
Where the edit helped, and where it did not
Qwen3-4B-Instruct-2507, original versus one edited matrix, on synthetic texts that must survive the trip into JSON without losing a character.
Final test · 120 cases by group
- Texts with tabsthe target failureOriginal19/30Edit30/30
- Line breaksalready perfectOriginal30/30Edit30/30
- Mixed special characterstabs, quotes, backslashesOriginal0/30Edit10/30
- Second table rowstill unsolvedOriginal0/30Edit0/30
Development, validation and controls
- Developmentused to fit the edit · not independentOriginal24/40Edit40/40
- Validationnew cases · 9 wins, 0 lossesOriginal10/40Edit19/40
- Random directioncontrol on validation · same strengthOriginal10/40Edit10/40
- Plain-text tabscontrol · real tabs must surviveOriginal10/10Edit10/10
Exact answer = after normal JSON parsing, the text field matches the source text character for character. Same prompt, temperature 0, seed and output limit for both models; saved and reloaded edit, no hooks, no grammar constraints, no repair. Final test, validation and controls frozen before any independent output of the edited model. Fifty of 120 final answers are still wrong.
Sources: our experiment records, 5 Nov 2026 (Lexassay research, internal): experiment 3 result report and final result by GPT-6 Astra working as a Codex agent (OpenAI). Synthetic test cases, not published.
Powered by Thorsten Meyer AI
A Narrow Gain in Data Formatting
The result points to a possible way of changing a model’s behavior without retraining all of its parameters: a targeted weight edit may help with a specific, repeatable formatting failure. Exact character preservation matters in software that passes structured data between services, because a malformed escape or altered value can make otherwise valid information unusable. This experiment, however, tested synthetic examples, not a production system or a broad range of JSON tasks.
The limits are visible in the reported results. Fifty of 120 answers remained wrong, and a separate task extracting a table row scored zero of 30 both before and after the edit. The finding is evidence of improvement on one test, not proof that the edit makes the model more reliable generally or that it will transfer to real-world workloads.
Products Worth Considering
Sony WH-1000XM5 Premium Noise Cancelling Wireless Headphones – Midnight Blue
Amazon Kindle Paperwhite (2024, 16 GB) – 7-inch Glare-Free Display, Black
Logitech MX Mechanical Wireless Illuminated Keyboard, Tactile Quiet, Graphite
As an affiliate, we earn on qualifying purchases.
The Earlier Qwen Editing Test
The experiment was part of a series of work on editing model weights. In an earlier test on Qwen, GPT-6 Astra tried to improve extraction of fields such as email addresses and due dates from synthetic business messages. The report says that edit showed early gains on validation examples, but on the frozen final test both the original and edited versions scored 97 of 200.
That earlier test also found 47 invalid answers in each version, mostly involving raw tab characters in JSON strings. The experiment report says this failure informed the next test. The latest work was therefore a targeted attempt to address a previously observed problem, rather than a general search for weaknesses across the model.
The site describes the work as its third experiment in a week involving AI systems examining or changing another model’s weights. Anthropic’s Claude was involved in an earlier experiment and reviewed that work, but the site says Claude did not take part in this Qwen edit. It says a Claude agent later researched and drafted the article about the results.
“The starting hint for experiment 3 “came from the previous attempt”.”
— ThorstenMeyerAI.com’s experiment report
Independent Checks Are Still Pending
The published figures are the experiment’s own results; the report says no second AI has reviewed this experiment. The article does not provide an independent replication or external assessment of the test design, the edit, or the analysis. The test cases are synthetic, and the reported gain does not establish how the edit would perform on naturally occurring text or other JSON formats.
It is also unclear whether the improvement holds across different prompts, model runs, or broader collections of inputs. The report gives the final score and says the test was frozen before the result was seen, but the supplied account does not establish that the change generalizes beyond this particular task. The separate table-row result remained 0 of 30, leaving no evidence of wider improvement from this edit.
Replication and Broader Testing
The immediate next step is independent scrutiny of the test and the saved weight change, which the publisher says has not yet occurred. The experiment runs are public on lexassay.com, according to the November 9 update, giving other reviewers material to inspect. A replication could test whether the reported gains persist with new cases and independent evaluation.
Further tests would also need to establish whether the edit helps on real data and whether it affects other model behavior. Until those checks are reported, the result is best understood as a measurable improvement on one frozen, synthetic JSON-escaping test—not as a general reliability upgrade.
Key Questions
What changed in the Qwen model?
The GPT-6 Astra Codex agent made a rank-one update to one matrix: the output projection of Qwen3-4B-Instruct-2507’s final MLP layer. The report says the other 397 stored tensors remained bit-identical.
How much did the JSON escaping score improve?
The edited model scored 70 of 120 exact answers on the frozen final test, compared with 49 of 120 for the original. That is 21 additional correct cases, or a 17.5-percentage-point difference on this test.
Did the edit improve every tested task?
No. The report says table-row extraction remained at 0 of 30. The latest improvement concerned copying special characters into JSON exactly.
Has anyone independently verified the result?
The publisher says the experiment has not yet had an independent review. Its runs are public on lexassay.com, but the supplied report does not describe an external replication.
Does this show that weight editing works generally?
No. The finding concerns one model, one targeted edit and a synthetic test. The report does not establish that the gain transfers to real-world data, other tasks or different prompts.
Source: ThorstenMeyerAI.com
