The article makes general assertions about systems. It gives plumbing, software, and machine learning as examples, and I simply add the examples of human learning and human societies.

An agent specifying the criteria of a test inspired by the real world and then evaluating the performance of a bunch of other agents is equally information-incestuous when those agents are LLMs as when they are humans. If the observations of the article must hold true for LLM training, then they must hold true for humans. If they do not hold true for humans, then they do not prohibit the existence of LLM training programs that likewise avoid the decoherence trap.

Given the fact that evolution was able to find human creativity and art as forms of cohering enrichment of human performance beyond the limits of the training data of a world without art, I would expect that there is a good chance tens of thousands of engineers and billions of dollars of compute will be able to find some form of abstract expression and learning as a form of cohering enrichment of LLM performance beyond the limits of the training data of a world without that abstract LLM expression.