If you don't allow Prisoners to vote, then there is incentive by a party to arrest the opposition forces, thereby reducing votes against them.

Its why arresting peaceful protestors and charging them is alarming.