Thanks, this is a really useful suggestion. On the 300-start panel, rich E #3 fixes 35.5% of the unedited policy's failures (33/93) and breaks 11.6% of its successes (24/207). Its same-norm random edit fixes about as many, 32.3%, but breaks far more, 27.1%. Across all six rich edits the averages are 21.5% fixed and 9.7% broken. Their random controls come out almost the same, at 20.1% and 10.4%.
I am still trying to learn and i'm guessing llm can't reliable thinker here and i am working on a larger study/paper
but from your point i got that failures are fragile. every edit, random ones included, flips a good share of failures into successes. That explains why the main paper practice checks, which were mostly on failures, looked like progress. Second, E #3 isn't better at repairing than random; it's better at not breaking things. We'll add both rates to the paper. Thanks again!