Ask Lemmygrad

I find by themselves the models, especially current-gen ones, are pretty bad at editing text. They still don't really grasp what it entails lol, because they are not aware of their limitations. And it seems that current models are trained mainly for technical (coding) tasks over anything else, so I feel it's only going to get worse in those applications.

But I've had some success using a test suite afterwards to confirm data integrity. Counting lines is one such method: you just compare the number of lines between the before and after and it gives you an idea of how much was cut off, but it's basic. An LLM in agentic can set up a full test suite to really understand what changed or not statistically, and then is able to bring back stuff from the older revision to ensure integrity and that it didn't do too much. There's a lot of other things it can use to test the data, and you can ask it for cross-tests too: two different tests that test the same thing, but do it in two completely different ways (like calculating "x*x" and then "x^2").