I think it's important to downvote them and even make them feel unwelcome. You can't let Nazis and white supremacists in your communities because any community that doesn't kick out Nazis will eventually find itself overrun by Nazis fleeing communities that kick them out.
You don't need to argue or make a fuss, but make sure to downvote anyone who doesn't believe all people are born equal and that people have a right to choose what to do and who to love.