They had to retract three of the findings almost immediately.

That tells me you can’t rely on whatever model they are using and instead they’re more likely just generating a much larger set of (mostly wrong) random results and then trying to filter out the invalid ones.

The dump is large enough that there should probably be some legitimate results in there, but that’s just emulating a million monkeys hammering away at a million keyboards.

The math dump has affected me less than the reverse-engineered Adobe suite and the flood of Opus-animated videos.

Pretty interesting and if anything out of it is real, then it's a significant breakthrough in machine learning. It means we have computer automation to automate research itself, which hopefully should amplify technological progress and not just replace workers, but it's concerning that we have to consider that impact in the first place.

Someone else said it's like a "million monkeys hitting a million typewriters". That's incredibly dismissive, but I think there's also a sliver of truth to it.

The randomness and hallucination/error rate is a spectrum here. It's not 100% random like a million monkeys hitting a million keyboards, but there's still some hallucinations that lead to big mistakes.

You see the same thing in computer security. There's a ton of "slop vuln reports", but now and then, one is real. The fact that generative AI can generate a real one now and then means that it's actually doing something interesting and powerful, yet that doesn't mean we'll necessarily see it hit 100% ever. I think we have to accept that it generates slop by design and someone needs to sift through slop in all various forms.

I think the AI oligarchs are realizing, and the rest of us are slowly seeing, is that you don't get exponential improvement by adding more datacenters. Maybe sigmoid or logarithmic. And we might not be able to get enough training data to make the AGI everyone dreamed of.

But there might be this middle ground where humans still actually train and work in these fields and use AI to help automate some aspects like today and sift through some slop when applicable and we greatly accelerate technological progress. This middle ground is not what they promised investors, and they're going to cause a financial crisis when investors realize it's slowing down to a halt, and they still have to pay employees.