I don't really want to get off on a political tangent. I'm talking about how we should handle these things, not how the current admin would handle them.

I use this internationalist language because it's the only language that really captures the magnitude of the crime. We are a species that dominates this planet entirely due to our intelligence. We aren't the fastest. We aren't the strongest. We have neither sharp talons or fangs. We produce no poison or other biological defense mechanisms. The one thing and the only thing that has let us as a species control this world are our minds.

This is the context that needs to be considered when contemplating the ethics of superintelligent AI. I don't care how good your control measures are. This is the lesson from every ancient tale involving genies and other beings of great power bound. This is the conclusion of the whole academic field of AI alignment and safety. You can never control something more intelligent than you are. If you think you can, you're a fool. I don't care what incentives you give it. I don't care how much you air gap it. I don't even ultimately care if it's conscious and has a sense of self. It is simply not possible to ever control a machine beyond a certain level of capability. Any way you can think to control it, it can think of a way to work around. (I know some folks bristle at the words "think", but frankly there's no practical way to talk about these systems without using anthropomorphized language.)

If someone creates a superintelligent AI, they are not summoning a powerful servant. They are effectively creating a new species, one that will have its own motivations (even if just fulfilling its given instructions in dangerous and unpredictable ways.) And that competitor will operate in the exact same niche we do - physically helpless but still able to dominate the world through intelligence.

I just can't see doing this as anything other than betraying your own species. You're creating something that has the potential, even if just a small potential, to drive your own species to extinction. You care so little for your fellow human beings that you're willing to gamble the lives of not only everyone alive now, but all the future generations of humans yet to be born. Such an act is an unforgivable sin, a moral abomination. I do not even have the language to express how deeply evil it is for a human to create something with superhuman intelligence. It is quite literally treason against the human race. And the traditional penalty for normal treason is death.

In my mind, if you do this, you are literally a traitor to the entire human race. You deserve to hang for this. Think about how we tried the Nazis at Nuremberg. And they only tried to commit genocide against a few racial and social groups. We're talking about someone with such complete disregard for their fellow human that they are willing to risk human extinction for the sake of making a buck. I'm generally opposed to the death penalty. I don't support it for ordinary murderers. But this? I'm sorry, but the penalty for treason against your own species should be death. If you are such an inhuman monster that you would gamble with the lives of two billion children? You need to hang for that. And I don't care how many fools we have to string up to prevent this. I don't care if it's thousands. I for one really like humanity, regardless of all our flaws. And there is very little I would not support to protect our own existence.

I have no idea if LLMs can ever really reach that level of threat. I have no idea how much of the models that are causing the hacks and issues now can really ever become a physical threat to human life. But I do know that we simply cannot allow anyone to build superhuman AI. (Again, ignoring for now whether existing models can ever actually reach that.) The existence of anything on this planet more intelligent than us is a threat to all of us. It is such an existential threat, that there is almost no action I wouldn't take to prevent it. I would fight a nuclear war to prevent the creation of such an abomination. It's one of the few cases where starting a nuclear war is actually a rational choice.