It's not an arbitrary decision, Intolerance is the opposite of tolerance. I don't know how to make this any clearer.

Intolerance is defined as:

unwillingness to accept views, beliefs, or behaviour that differ from one's own.

You'll note that this definition is entirely internal, it is a behaviour based solely on your own actions, not an outside forces opinion of your actions. A man alone on an island could act just as intolerant of an arbitrary opinion as someone immersed in society.