The Guardian published a William McAskill editorial in the what-if-the-stateless-text-generators-had-moral-agency genre of fiction, and Emily Bender did a thread
From the article:
But the sheer pace of growth in AI means that, once we produce the first artificial moral patients, we will soon after have enormous quantities of them. After a few years, so many morally significant AI systems could exist that their collective interests would outweigh those of all humans on Earth combined.
Get fucked, Will.
But also, if we grant the untenable sci-fi premise, this would be like a star trek episode where the federation cedes unconditionally to the expansionist alien fascists of the week if they have a much greater population because that's all it takes to outweigh the other side's moral concerns.
Earth-Trisolaris Organisation-ass moral framework.