All of us are trained on stolen data too. Unless you invented your own language and came up with every idea you have from scratch, then we're all running on borrowed information. It's our shared inheritance as humanity.

That's why open-weight local models are the only way LLMs should be done. The data belongs to us, humanity. We made the data that they scraped, over thousands of years, and we're still doing it to this day every time we post a comment on a public forum.

It shouldn't be kept locked up in a vault where some oligarch can control its distribution and divvy it out for profit. It should be held in a public repo where anyone can download it, run it, tweak it, like any other open-source software. It belongs in the public domain.

Diffusion generation is different of course because the data is more complex, and stylistic details are often directly spoofed from the originals. It's taking magazine-cuttings and calling them original work. AI slop isn't art.

Language generation on the other hand is just... how humans have been speaking and sharing ideas since we began verbally communicating.